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ABSTRACT 


This thesis develops a stochastic representation of a tactical commander’s 
decision cycle and applies the model within the high-resolution combat simulation: 
Combined Arms Analysis Tool for the 21st Century (Combat XXI). Combat XXI is a 
Joint Army-Marine Corps effort to replace the Combined Arms and Support Evaluation 
Model (CASTFOREM)—a legacy combat simulation. Combat XXI is a non-interactive, 
high-resolution, analytical combat simulation focused on tactical combat. Combat XXI is 
being developed by the U.S. Army TRADOC Analysis Center-White Sands Missile 
Range (TRAC-WSMR) and the Marine Corps Combat Development Command 
(MCCDC). Combat XXI models land and amphibious warfare for applications in the 
research, development and acquisition, and the advanced concepts requirements domains. 
Stochastic decision-making enhances Command and Control (C2) decision processes in 
Combat XXI. The stochastic simulation of a commander’s decision cycle (SSIM 
CODE) addresses variability in decision-making due to uncertainty, chance and the 
commander’s attributes. A Bayesian Network representation of a conditional probability 
model for a commander’s decision cycle is implemented in SSIM CODE. This thesis 


develops, applies and evaluates the effectiveness of SSIM CODE. 
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EXECUTIVE SUMMARY 


This thesis develops a representation of a tactical commander’s decision cycle and 
implements it in a computer simulation. A stochastic decision cycle model is applied 
within the high-resolution combat simulation: Combined Arms Analysis Tool for the 21* 


Century (Combat XXI). 


The thesis objectives include: 
" Model tactical commander decision cycles (battalion and below). 
® Apply command and control (C2) doctrine. 
® Develop a functionality module for Combat XXI. 
® Exercise the stochastic simulation of a commander’s decision cycle 


(SSIM CODE) as a stand-alone simulation. 
® Evaluate the effectiveness of SSIM CODE’s decision-making. 


Combat XXI is a Joint Army-Marine Corps effort to replace the Combined Arms 
and Support Evaluation Model (CASTFOREM)—a legacy combat simulation. Combat 
XXI’s charter includes meeting or exceeding CASTFOREM’s capabilities. Combat XXI 
is a non-interactive, high-resolution, analytical combat simulation focused on tactical 
combat. Combat XXI models land and amphibious warfare for applications in the 
research, development and acquisition, and the advanced concepts requirements domains. 
Combat XXI is being developed by the U.S. Army TRADOC Analysis Center-White 
Sands Missile Range (TRAC-WSMR) and the Marine Corps Combat Development 
Command (MCCDC). These agencies seek to incorporate C2 decision-making with an 


appropriate degree of realism in Combat XXI. 


X1x 


C2 in CASTFOREM is accomplished using an expert system that refers to a 
knowledge base. The knowledge base is a set of decision tables that prescribe decision 
outcomes according to expert judgment. One of the major assumptions in 
CASTFOREM’s C2 module is that tactical “Decision processing takes no [simulation] 


time.” (TRAC-WSMR, 1999) 


The analysis requirements driving Combat XXI's development call for an 
enhanced representation of the commander and the his decision process. The C2 
component in Combat XXI can be enhanced by a decision-making model implemented as 
a functionality module (an interface by which Combat XX] accesses services and specific 
combat processes such as movement, communications, and engagement). SSIM CODE 
(a Combat XXI functionality module for stochastic, tactical decision-making) addresses 


variability in decision-making due to uncertainty, chance and a commander’s attributes. 


The key facets of simulating decision-making in C2 include: representing the 
complete commander’s decision cycle, portraying the evolving nature of the 
commander’s awareness, and capturing the stochastic nature of decision-making due to 


uncertainty and chance. These attributes are included in SSIM CODE. 


The SSIM CODE model builds on three basic elements: an Observe-Orient- 
Decide-Act (OODA) loop-based decision cycle, dynamic situational awareness, and 
stochastic decision-making. The functionality of the SSIM CODE is based on the OODA 
loop. The Combat XXI situational awareness (SA) module structure is used by SSIM 
CODE for dynamic SA. A Bayesian C2 network provides stochastic decision-making in 


SSIM CODE. 


XX 


SSIM CODE is programmed in Java. The use of Java allows the development of 
an object-oriented, event-driven model that meets Combat XXI requirements for a 
functionality module. To meet the Combat XXI functionality module requirements, 
SSIM CODE must implement the methods (subroutines or processes) specified by the 
Combat XXI functionality module interface. SSIM CODE development and testing 


includes over nine-thousand lines of Java code. 


Combat XXI and SSIM CODE use Simkit as a simulation engine. Simkit is a 
Java class library (collection of Java programs) for event-driven, component-based 
simulation. Figure 1 depicts the Combat XXI/SSIM CODE relationship. Because SSIM 
CODE must interact with Combat XXI as the simulation runs, SSIM CODE must be 
capable of placing Simkit events (SimEvents) on the Simkit event list and monitoring 


state variable changes from the Combat XXI simulation. 
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Figure 1. Combat XXI/SSIM CODE Relationship 
SSIM CODE?’s commander entity is a Combat XXI functionality module that interfaces with 
the rest of the simulation through the SA module. 
XX1 





Decision factors are binary, discrete random variables computed as functions of 
varying states in the combat simulation. Decision factors are aggregated elements that 


influence tactical decision-making. 


In practice, commanders make decisions based on reported estimates—not on 
perfect information. To model this concept, report nodes are used with decision factors 
in a Bayesian network. Three sets of nodes are used: the commander’s decision, reports, 
and decision factors. The lack of perfect information in tactical decision-making is 
captured in the relationship between the three sets of nodes. The decision outcome is 
probabilistically dependent on report states, and it is independent of decision factor 


states. Figure 2 shows a Bayesian network with imperfect information. 
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Figure 2. Bayesian Decision-Making Network (After Stephens, 1998) 
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The report nodes represent uncertainty inherent to the commander’s information. 
Based on the Bayesian network in Figure 2, the commander’s decision is conditionally 
independent of E, R and B, given R;, R2 and R3, SSIM CODE is capable of collecting 


information from the Combat XXI simulation to develop reports for the commander. 


The SSIM CODE model is centered on the commander entity. A commander’s 
individual characteristics are considered in the SSIM CODE’s decision-making process. 
The SSIM CODE commander entity possesses an SA module, a C2 style, a C2 


philosophy, an experience level, and a set of decision cycles (OODA loops). 


SimEvents from within Combat XXI trigger changes in the SA module’s facts. 
The commander entity in SSIM CODE monitors these changes. When a decision is 
required, the appropriate type of OODA loop is started. Reports on decision factors are 
received and a perception of the current situation is developed. The Bayesian network is 
used to determine a decision outcome. The decision is then implemented with a set of 


actions. The SA module’s facts are updated, and subsequent decisions are scheduled. 


Two stages of fractional factorial design experiments are used in evaluating SSIM 
CODE. SSIM CODE is deemed to make tactical sense through a face validation. The 
evaluation concludes that the first steps in developing a decision-making model for 


Combat XXI and the purpose of this thesis are accomplished. 


SSIM CODE has applications within Combat XXI and other Department of 
Defense simulations. The Australian armed forces will also be replacing CASTFOREM 
with Combat XXI. Improved C2 processes from SSIM CODE can serve to enhance 


Combat XXI applications in both U.S. and Australian modeling and simulation domains. 
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I. INTRODUCTION 


A. THESIS PURPOSE 

This thesis develops a representation of a tactical commander’s decision cycle and 
implements it in a computer simulation. A stochastic decision cycle model is applied 
within the high-resolution combat simulation: Combined Arms Analysis Tool for the 21* 


Century (Combat XXI). 


An approach to developing a decision-making model for Combat XXI includes: 


® Develop the concept of tactical decision-making for command and control 
(C2) into an analytical model. 


" Implement the decision-making model in a simulation loosely coupled with 
Combat XXI’s behaviors package. 


® Evaluate the performance of the decision-making simulation compared to 
the analytical model. 


® Link the simulation to all applicable Combat XXI modules (tightly coupled 
with Combat XX1). 


® Enhance the abstract features of the simulation to handle all likely 
applications of Combat XXI1. 


This thesis accomplishes the first three steps of this approach. An analytical, 
stochastic decision-making model is developed. The model is then implemented in a 
simulation that is loosely coupled with Combat XXI. Finally, the model is evaluated with 
a test scenario. The thesis objectives and scope are discussed at the end of Chapter II. 


B. DECISION-MAKING IN COMBAT SIMULATIONS 
1. The Need for a Stochastic Decision-Making Model 


The Panel on Modeling Human Behavior and Command Decision Making was 
formed by the National Research Council in 1996 to evaluate human behavior 


representation in military simulations (Stephens, 2000). This panel conducted an 


1 


eighteen-month study that included an in-depth evaluation of decision-making in combat 


simulations. 


According to the panel’s 1998 report, most combat simulations assume no 
variability in decision-making. These simulations apply scripted or deterministic 
decision-making processes and fail to provide the necessary realism in decision-making: 

The Under Secretary of Defense for Acquisition and 
Technology has set an objective to “develop authoritative 
representations of individual human behavior’... Yet...users 
of military simulations do not consider the current 
generation of human behavior representations to be 
reflective of the scope or realism required for the range of 


applications of interest to the military. (Pew and Mavor, 
1998) 


The intrinsic randomness in human decision-making must be represented with a 
stochastic decision-making model. This thesis focuses on the tactical commander. The 
thesis develops, implements, and evaluates a stochastic tactical decision-making model. 


2. The Battlespace’s Influence on Tactical Decision-Making 


The Marine Corps Combat Development Command (MCCDC) is modeling 
battlespace phenomena that influence decision-making. These areas include non- 
linearity, intangibles and co-evolving landscapes. Non-linear effects occur when minor 
actions can have large impacts on combat outcomes. An example is the receipt or non- 
receipt of a single message that changes the outcome of an entire battle. Intangible 
factors include morale, training, leadership-style, command philosophy, etc. The co- 
evolving landscapes concept describes a setting where commanders on both sides apply 


their decision-making in anticipation of each other’s actions. (Brandstein, 1999) 


These three phenomena impact tactical decision-making in the battlespace. 
Representing these features of warfare contributes to realism in a decision-making 
simulation. An effective decision-making model should contribute toward the depiction 


of these sources of realism. 


The Combat XXI simulation is currently being co-developed by the U.S. Army 
TRADOC Analysis Center at White Sands Missile Range (TRAC-WSMR) and MCCDC. 
These agencies seek to incorporate an appropriate degree of realism in C2 decision- 
making within Combat XXI. A stochastic decision-making model that contains 
representations of non-linearity, intangibles and co-evolving landscapes would contribute 
toward an enhanced C2 decision process in Combat XXI. 


C. THE COMBAT XXI SIMULATION 


Combat XXI models land and amphibious warfare for applications in the 
Research, Development and Acquisition (RDA), and the Advanced Concepts 
Requirements (ACR) domains. Combat XXI is a non-interactive, high-resolution, 
analytical combat simulation focused on force-on-force tactical combat (brigades, 
battalions and below). Combat XXI is a Joint Army-Marine Corps effort to develop a 
replacement for the Combined Arms and Support Evaluation Model (CASTFOREM). 
CASTFOREM is a legacy combat simulation used to represent combined-arms ground 
combat. CASTFOREM is a_ high-resolution, two-sided, stochastic, closed-loop 


simulation. It has been in use for over fifteen years. (TRAC-WSMR, 1999) 


Combat XXI is composed of discrete software packages (collections of 


programs). Component packages are reusable programming elements. Some of these are 


Combat XXI proprietary packages, and others are extensions to components developed 


independently of Combat XXI. (Olson, 2000) 


Figure 1 shows the hierarchy of Combat XXI packages. Foundation packages 
provide key services and base objects used throughout Combat XXI. Examples include a 
simulation engine, data base connectivity, and random number generation. Core 
packages provide more precise functions by building upon foundation packages. These 


functions include scenario input/output, terrain services, and data logging. (Olson, 2000) 


A final layer of abstract services is added by functionality packages that build 
upon the core and foundation packages. Integration packages combine abstract services to 


accomplish tangible tasks in the context of a study. These tasks include scenario 


definition, movement, search and acquisition, and engagement. (Olson, 2000) 
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Figure 1. Combat XXI Component Packages (After Olson, 2000) 
Each discrete software package builds on the layers below. 
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1. C2 in CASTFOREM 


Figure 2 is an overview of CASTFOREM’s structure. CASTFOREM’s unit of 
resolution (UOR) is an individual tank, vehicle or other combat platform. A 
CASTFOREM UOR can have six physical processes (move, engage, search, 
communicate, engineering and combat service support) and a C2 process. C2 in 
CASTFOREM is accomplished using an expert system that refers to a knowledge base. 
The knowledge base is a set of decision tables that prescribe decision outcomes according 
to expert judgment. One of the major assumptions in CASTFOREM’s C2 module is that 


tactical “Decision processing takes no [simulation] time.” (TRAC-WSMR, 1999) 








Command 


and 
Control 


_ Decision 
Tables 








Figure 2. CASTFOREM’s Functionality Structure (From TRAC-WSMR, 1999) 
Unit functionality consists of six physical process and C2. 


Decision tables are invoked as a result of simulation events in CASTFOREM. 
Each UOR updates it’s situational profile (set of ‘known’ facts) when specific simulation 
events occur. Based on the knowledge base rules and a UOR’s situational profile, the 


decision tables generate a set of primitive orders (move, engage, search, communicate, 


) 





etc.) that comprise a UOR’s course of action. Random outcomes are included in 
CASTFOREM. The variability of these stochastic outcomes depends directly on the 


extensiveness of the decision tables. (TRAC-WSMR, 1999) 


Expanding or decreasing available options in the knowledge base changes 
decision variability in CASTFOREM, as illustrated in Figure 3. The specific variability 
desired and the adjustments to the decision table knowledge base must be established 


before simulation run-time. (TRAC-WSMR, 1999) 
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Figure 3. CASTFOREM Decision-Making Variability (From TRAC-WSMR, 1999) 
Variability is controlled by the number of options available in the knowledge base and 
their associated probabilities. 


2. C2 in Combat XXI 


Combat XXI’s_ charter includes meeting or exceeding CASTFOREM’s 
capabilities. Combat XXI is being developed in Java; CASTFOREM is programmed in 
SIMSCRIPT. The object-oriented nature of Java, it’s platform independence, the 


available open-source Java tool kits, and Java’s package-based component structure 





provide Combat XXI significant flexibility and potential for expansion. Combat XXI 
should exceed most of CASTFOREM’s capabilities. The analysis requirements driving 
Combat XXI's development call for an enhanced representation of the commander and 


the command decision process. 


The goals for C2 behaviors in Combat XXI include “...modeling the commander’s 
view of the battlefield and the decision logic that the commander would use to determine 
a course of action.” (Harless, 2000) The C2 component in Combat XXI can be enhanced 
by a decision-making model implemented as a functionality module (an interface by 
which Combat XXI accesses services and specific combat processes such as movement, 
communications, and engagement). The stochastic simulation of a commander’s decision 
cycle (SSIM CODE) developed in this thesis seeks to fill that role. 


D. MODELING TACTICAL DECISION-MAKING 


Forming a tactical decision-making model begins with defining the commander’s 
decision-making as it relates to C2. Commanders are central to the C2 process and make 
the vital decisions in the battlespace. They make informational decisions (what is 
happening?), operational decisions (what actions should be accomplished?) and 
organizational decisions (how should forces be arranged?) (Orr, 1996). A C2 model 


should focus on the commander and his decision cycle. 


The commander's perception is the pivotal part of his decision cycle (Boyd, 
1995). A tactical decision-making model should thus include: a representation of the 
commander’s decision-making process, an emphasis on his perception, a portrayal of 


uncertainty and chance, and a decision cycle structure. 


1. The Commander’s Decision-Making Process 


A commander’s decision-making begins as an intuitive process. At the initial 
stage of decision-making, neither the current situation nor the desired end-state may be 
fully apparent. The commander formulates his objectives based on directives from 
higher-headquarters. He formulates an understanding of the measures required to 


accomplish his mission. 


By gathering information on the battlespace, the commander clarifies his image of 
the current situation. He then develops several alternatives or courses of action (COAs) 
for reaching his desired end-state from the current situation. Finally, the commander 
reaches a decision and selects a plan to accomplish his objectives. Figure 4 summarizes 
this process. The commander’s decision-making process is continuous. He revisits and 


updates his decisions, as the dynamic situation requires. 
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Figure 4. Commander’s Decision-Making Process (After Orr, 1996) 
The commander clarifies the end-state or goal then chooses a means to attain the goal. 





2. Variability in Tactical Decision-Making 


The commander applies both an analytical methodology and his intuition to 
decision-making. Purely analytical decision-making usually produces consistent results 
in similar situations. However, the commander’s intuition introduces variation to the 
decision-making process. Variability in decision-making is in part due to the 
commander’s human nature. Specifically, the commander’s decisions are influenced by 


personal attributes. 


Uncertainty and chance contribute to further variability in the commander’s 
decisions. The specific information available to the commander for a given decision, the 
degree to which that information represents reality, and the commander’s interpretation 
of the information are all sources of uncertainty in C2. The complexity of the 
commander’s C2 system and the random interaction between the components of that 
system add more variability to the commander’s decisions. It follows that a stochastic 
model is required to represent the variability in tactical decision-making. 

3. The Commander’s Perception 

An essential element of tactical decision-making is the mental image that 
represents the commander's "knowing and seeing" (Kahan, Stasz and Worley, 1989). 
The commander's perception is an estimate of reality influenced by his individual 
attributes and by the information he collects. A commander builds this perception by 
evaluating his mission, the enemy, his troops, the terrain, the weather, and the time 
available (METT-T) (U.S. Marine Corps, 1996). Commanders are taught to 
conceptualize the battlespace in terms of METT-T through doctrine and training (Kahan, 


Stasz and Worley, 1989). 


Tactical decision-making is the process of transforming the commander’s 
perception into action. Figure 5 summarizes this process. The commander’s image is 
influenced by his current view of the battlespace. His assigned mission, guidance from 


superiors, training, and individual attributes also shape his image. 
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Figure 5. Translating an Image into Action (After Kahan, Stasz and Worley, 1989) 
Various elements influence the commander’s image. His decision cycle transforms the 
image into action. 

4. Information-Processing Styles 

Different information-processing styles determine how and when the commander 
employs his decision cycle. A study by the RAND Arroyo Center (a U.S. Army research 
and development center) on commanders’ information needs concluded that three 
information-processing styles are employed by military commanders in decision-making: 
directed (one-way), triggered, and inquiry-based information-processing (Kahan, Stasz 
and Worley, 1989). | These information-processing styles determine how the 


commander's knowledge and perception are developed. SSIM CODE’s representation of 


each of these information-processing styles is discussed in Chapter IV. 
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Directed information-processing involves the presentation of information to the 
commander in a set order. Decisions are made according to time constraints since a 
complete set of information may not be attainable (Kahan, Stasz and Worley, 1989). 


Figure 6 illustrates directed information-processing. 
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Figure 6. Directed Information-Processing (After Kahan, Stasz and Worley, 1989) 
Information is received in a sequential order before the decision outcome is reached. 


In triggered information-processing, certain events or thresholds initiate the 
commander’s decision-making. The commander defines what critical information will 
indicate that a decision is required (Kahan, Stasz and Worley, 1989). Commander’s 
critical information requirements (CCIRs) represent these triggers. CCIRs are 
information needs identified by the commander regarding enemy forces, friendly forces 
and the environment. CCIRs are critical to timely decision-making (MSTP Staff, 2001). 


Figure 7 depicts triggered information-processing. 
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Figure 7. Triggered Information-Processing (After Kahan, Stasz and Worley, 1989) 
Key events trigger decision-making as they occur. The commander determines which 
events will act as triggers or CCIRs. 


Inquiry-based information-processing is a demand-pull approach to developing 
the commander’s knowledge. When the commander determines that a decision is 
required, he makes inquiries about specific information. Figure 8 is a representation of 


inquiry-based information-processing. 








Make Decision X 


| 









Obtain Information A 








Obtain Information B Make Decision Y First 





Figure 8. Inquiry-Based Information-Processing (After Kahan, Stasz and Worley, 1989) 
Making one decision leads to collection of information and possibly other decisions that 
must be resolved first. 


12 








The information-processing style applied by a commander is influenced by his 
leadership style; however, the same commander may use each of the three styles or a 
hybrid method. Information-processing influences a commander’s perception—the key 
element in his decision-making. A tactical decision-making model must be able to 
represent each of these information-processing styles. 

5. The Commander’s Decision Cycle 

The tactical decision-making process is a cycle repeated continuously by the 
commander. Colonel John R. Boyd’s Observe-Orient-Decide-Act (OODA) loop is a 
concise model of a commander’s decision cycle. A military commander first forms an 
observation of the battlespace through communications, sensors and intelligence systems. 
Next, he processes observed information to develop his perception as a frame of 
reference. Based on his orientation, the commander then makes decisions to attain his 
mission objective. Finally, those decisions result in actions, which influence the 
battlespace. Subsequent observations initiate further iterations of the OODA loop. 


(Boyd, 1995) 


The OODA loop encapsulates the decision-making process and includes a 
representation of the commander's perception in the orientation phase. The OODA loop 


provides a suitable general structure for a tactical decision-making model. 
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I. BACKGROUND 


A. ELEMENTS OF A DECISION-MAKING SIMULATION 

The key facets of simulating decision-making in C2 include: representing the 
complete commander’s decision cycle, portraying the evolving nature of the 
commander’s awareness, and capturing the stochastic nature of decision-making due to 


uncertainty and chance. These attributes are desired in a tactical decision-making model. 


Techniques for simulating these intangible combat phenomena have been 
developed by several modeling and simulation organizations. Previous simulation 
modeling efforts (described below) are used in the development of SSIM CODE. The 
SSIM CODE model builds on three basic elements: stochastic decision-making, an 
OODA loop-based decision cycle, and dynamic situational awareness. 

1. A Decision Cycle Simulation 

The Command, Control, Communications, Computers and Intelligence (C4I) 
Modeling, Simulation, and Assessment Directorate of the Defense Information Systems 
Agency (DISA) has developed and implemented a C2 simulation model. This C2 model 
is an element of the DISA Joint C4I, Surveillance, and Reconnaissance (C4ISR) model 
(DISA, 2000). The DISA C4ISR model has been used in studies to support 


Commanders-in-Chief (CINCs) and the Joint Staff. 


The DISA C4ISR model is a federation of five interacting simulations: a combat 
model, a sensor model, a communications assessment model, an information model, and 
a C2 model. The DISA model is focused on the operational level. DISA’s model is a 


more aggregated representation of the battlespace than the tactically oriented Combat 
LS 


XXI. However, DISA’s C2 module effectively implements a commander’s decision 


cycle that has applications at all levels of warfare. 


The functionality in DISA’s C2 simulation fully encompasses the commander’s 
OODA loop. This C2 simulation is robust. It is capable of representing a decision cycle 
while interacting with other elements of a combat simulation. DISA’s C2 model has 
been tested in several analyses, including CINC operations plan (OPLAN) assessments. 
This C2 simulation is used to structure the functional requirements of SSIM CODE. 


2. A Dynamic Situational Awareness Module 


A methodology for modeling a commander’s Situational Awareness (SA) has 
been developed by TRAC-WSMR. Combat XXI implements an SA module construct. 


This structure represents the commander’s dynamic SA. 


The SA module “listens” to events and property changes (target detections, force 
movements, modifications to entity attributes, etc.) during a simulation run. The SA 
module then interacts with an expert system—a collection of facts, rules, and actions. 
This interaction between the SA module and the expert system results in prescribed 


actions if pre-defined conditions are met. 


The Combat XXI SA module fulfills the role of the decision table based expert 
system in CASTFOREM. Furthermore, the SA module is capable of dynamically 
changing the set of potential outcomes and actions during simulation run-time. The 
dynamic SA structure developed by TRAC-WSMR provides a means for the 
commander’s decision cycle in SSIM CODE to interact with other elements of the 


Combat XXI simulation. 
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3. A Stochastic Decision-Making Model 

Computing Technologies, Inc., with MCCDC’s Studies and Analysis Division, 
has developed an approach for simulating command and control as a behavioral model. 
An exploration report on C2 titled “Project Albert and JWARS,” (Stephens, 2000) details 
this approach and the application of a Bayesian joint-probability network to represent the 


stochastic results in C2. 


In MCCDC’s Bayesian C2 model, state variables from throughout the simulation 
(sensor module, combat module, etc.) are measured to determine decision factors. The 
decision factors are defined as binary random variables that generate variability in the 
commander’s decision-making process. The stochastic nature of the Bayesian C2 model 
is derived from these decision factors. This decision-making model is used in SSIM 
CODE. 

4. Combining the Decision Cycle Simulation Elements 

The functionality of the SSIM CODE is based on the DISA C2 model. The 
Combat XXI SA module structure is used by SSIM CODE to model dynamic SA while 
providing a means for interaction with other combat simulation elements. The MCCDC 
Bayesian C2 model provides a methodology for stochastic decision-making in SSIM 


CODE. 


The DISA C2 model, the Combat XXI SA module structure, and the MCCDC 
Bayesian C2 model are the primary sources for the design of SSIM CODE. These 


characteristics are described in detail in Chapter III. 
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SSIM CODE is designed as a Combat XXI functionality module. This design 
goal required the use of Java and Simkit (a Java-based simulation engine). The 
relationship between SSIM CODE, Java and Simkit are described in the following 
sections. 


B. SSIM CODE AND JAVA 


SSIM CODE is programmed in Java. Java is an object-oriented, platform 
independent programming language developed by Sun Microsystems.  Java’s 
characteristics support SSIM CODE’s objectives. Because SSIM CODE’s model 
structure is object-oriented and event-driven, Java is an appropriate programming 
language choice. More importantly, Combat XXI is being developed in Java. Therefore, 
the use of Java allows SSIM CODE to be developed as an object-oriented, event-driven 
model that meets the Combat XXI functionality module requirements described below. 


1. SSIM CODE as an Object-Oriented Model 


The object-oriented nature of Java allows for the creation of generic object 
templates, such as a commander. Commanders are modeled as individual entities or 
objects. Each object meets the generic description of its class (or type) with a set of basic 
properties. For example, a SSIM CODE commander entity always includes a command 
level, a C2 philosophy, a C2 style, an experience level, a set of OODA loops, etc. 
Specific characteristics individualize these objects. A specific individual commander 
entity is referred to as an instance of the commander object. (A detailed discussion of the 


commander attributes is provided in Chapter IIL.) 


Java objects can be nested: an object may have a property that is also an object. 


The commander entity has properties that are also objects, such as OODA loops. OODA 
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loops consist of a decision type, delay times between phases, and a reference to a specific 
instance of the commander object. ODA loops contain individual decisions as 
properties. These decisions are objects that are instantiated (created from a general class) 
when an OODA loop starts. Decisions consist of a decision type, a request time, a start 
time, report data, an end time, and a decision result. A decision object includes a 
reference to the OODA loop that instantiated the decision. Java’s object-oriented trait 
allows for the straightforward implementation of the SSIM CODE model into a computer 
program. 


2. SSIM CODE as a Combat XXI Functionality Module 


Java is a significant common feature shared by SSIM CODE and Combat XXI. 
The common programming language makes it possible to design SSIM CODE as a 
Combat XXI functionality module. Combat XXI implements several types of entities, 
such as platforms. Platforms are Java representations of vehicles and personnel. 
Functionality modules are components of platform instances. Functionality modules 
serve as process delegates for platforms in Combat XXI. Examples of processes handled 
by functionality modules on behalf of a platform are movement, search, communications, 


and engagement. 


To meet the Combat XXI functionality module requirements, SSIM CODE must 
implement the methods (subroutines or processes) specified by the Combat XXI 
functionality module interface. | These prescribed methods primarily ensure that a 
platform can employ its modules generically and without explicitly modifying the 
platform’s Java code for any particular module. For example, each module defines its 


type (e.g., "mobility") from a list of predefined values. 
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Extensions to the functionality module interface prescribe the methods associated 
with a specific type of module. The commander entity in SSIM CODE is a functionality 
module extension. Thus, the commander entity contains methods specified by the 
Combat XXI functionality module interface and specialized methods required to make 
decisions using a decision cycle. 


C. SSIM CODE AND SIMKIT 


SSIM CODE uses Simkit as a simulation engine. Simkit is a Java class library 
(collection of Java programs) for event-driven, component-based simulation. LtCdr Kirk 
Stork designed Simkit in his thesis: Sensors in Object Oriented Discrete Event 
Simulation (Stork, 1996). Professor Arnie Buss, at the Naval Postgraduate School, 
further developed Simkit as a Java class library. 

1. Simkit Modeling 

Simkit is a discrete event simulation tool. A process modeled by Simkit is a set of 
discrete events that occur according to a schedule or event list. The Simkit event list 
drives the discrete event simulation (Buss, 2000). For example, the activation of a sensor 
(initiated by an event) schedules the conduct of a search. When executed, the search may 
acquire potential targets and may initiate state changes in a targeting system. Simkit 
events (SimEvents) activate methods within Java objects invoked at a scheduled time to 


cause state changes in the model. 


Implementing a model using Simkit requires representing the system or process 
with simulation objects. The states and state transitions in each simulation object must be 
specified. State variables define a simulation object’s state at a specific time. SimEvents 


initiate property changes in state variables. For example, simulation objects may include 
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sensors and targets. The number of acquired targets may be represented in the state of 


the model. 


SimEvents define state transitions. As methods are invoked within a simulation 
object, the Simkit engine generates SimEvents and schedules them on the event list. At 
the appropriate (scheduled) time, a SimEvent is passed to the proper method, and the 
state changes included in the state transition are initiated. A SimEvent can schedule other 
SimEvents. The time order of events is maintained by the event list. 


2. Simkit Links Combat XXI and SSIM CODE 


Combat XXI uses Simkit as its simulation engine. Because SSIM CODE must 
interact with Combat XXI as the simulation runs, SSIM CODE must be capable of 
placing events on the event list and monitoring state variable changes from the Combat 
XXI simulation. Thus, SSIM CODE also employs Simkit. SSIM CODE is capable of 
collecting information from the Combat XXI simulation to develop reports for the 
commander. SSIM CODE places each individual phase of the commander’s OODA loop 
on the event list. Thus, delays within the commander’s decision process are included in 
the simulation along with all other time-consuming processes modeled by Combat XXI 


(such as movement, search, etc.). 


Java and Simkit are the major features shared by SSIM CODE and Combat XXI. 
These commonalities contribute to the loose coupling of SSIM CODE (the functionality 
module) and Combat XXI (the combat simulation). Figure 9 presents a simplified 
relationship between Combat XXI and SSIM CODE. The platform entity, SA module 


and functionality module interface are all elements (Java classes) of Combat XXI. The 


21 


commander entity is part of SSIM CODE and complies with the functionality module 


interface requirements. 


Simkit is the simulation engine for both Combat XXI and SSIM CODE. 
SimEvents link the SA module and the commander entity. The SA module monitors and 
schedules SimEvents through the use of facts and actions (described in the Model 
Structure section). The commander entity uses its OODA loops to monitor and schedule 


SimEvents. 
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Figure 9. Combat XXI/ SSIM CODE Relationship 
SSIM CODE’s commander entity is a Combat XXI functionality module that interfaces with the 
rest of the simulation through the SA module. 
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D. THESIS OBJECTIVES 


This thesis contributes toward the Combat XXI enhanced C2 decision process 
component by forming a representation of the tactical commander’s decision. The thesis 
objectives include: 

" Model tactical commander decision cycles (battalion and below). 

® Apply C2 doctrine. 

® Develop a functionality module for Combat XXI. 

® Exercise the SSIM CODE as a stand-alone simulation. 

® Evaluate the effectiveness of SSIM CODE’s decision-making. 


E. THESIS SCOPE 


This thesis develops, implements and evaluates SSIM CODE. SSIM CODE is 
loosely coupled with a fixed version of Combat XXI. Because Combat XXI is currently 
under development, its features and structure change daily. Certain essential features of 
Combat XXI (such as the engagement process) were not complete at the time SSIM 
CODE was being developed. For these reasons, evaluation of SSIM CODE’s 
performance is conducted with a stand-alone simulation. The evaluation simulation is 


coupled to Combat XXI through the SA module. 


Model assessment includes testing SSIM CODE with a combat scenario. The 
scenario centers on a company commander’s decision. The test scenario involves 
assumptions about the capabilities and characteristics of the forces involved. The 
assumptions include force structure, commander characteristics, offensive and defensive 
tactics, etc. The thesis analysis focuses on comparing SSIM CODE’s performance to the 
analytical models developed in Chapter III. The evaluation also involves the use of 


quantitative MOEs that represent a commander’s intent as discussed in Chapter V. 
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Analysis of SSIM CODE also involves a face validation (U.S. Army, 1999). A 
discussion of the requirements in a rigorous validation of a simulation, such as SSIM 
CODE, is included in Chapter VII. However, a full validation of SSIM CODE is not 


within the scope of this thesis. 
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HI. MODEL DEVELOPMENT 


SSIM CODE’s characteristics include functionality based on the DISA C4ISR C2 
model, a basis in Marine Corps C2 philosophy, stochastic decision-making modeled by 
the MCCDC Bayesian network, and the capability to interface with the Combat XXI SA 
module structure. 


A. MODEL FUNCTIONALITY 


Based on DISA’s C4ISR model, SSIM CODE’s functionality is structured 
according to the OODA loop. The elements in each OODA loop phase include: 


" Observe 
— Get Combat State Data. 
— Receive Reports. 

= Orient 
— Fuse Report Data to Develop Decision Factors. 
— Develop a Combined State Perception. 

= Decide 
— Apply Decision Factors to the Decision Process. 
— Choose a COA. 

" Act 
— Develop a Set of Commands to Represent the COA. 
— Issue Commands. 


B. C2 PHILOSOPHY 

Marine Corps C2 doctrine describes two C2 philosophies: detailed C2 and 
mission C2. Detailed C2 pursues certainty while minimizing uncertainty. Detailed C2 is 
analytical, centralized and technology intensive. Mission C2 accepts uncertainty and 
risks. Mission C2 is a decentralized, flexible process that relies on lower-level decision- 


making. (U.S. Marine Corps, 1996) 
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The philosophy behind mission C2 views uncertainty as an unavoidable product 
of war that cannot be eliminated. While mission C2 calls for reducing uncertainty, its 
focus is on generating a rapid tempo. Reducing uncertainty involves the timely process 
of collecting and processing information. Speed is a key element of mission C2. 


Therefore, in mission C2 tempo is not sacrificed to eliminate uncertainty. 


Detailed C2 is based on the idea that nearly all information in the battlespace is 
ultimately available. The focus of detailed C2 is eliminating uncertainty through superior 
information-processing. Tempo in detailed C2 is derived from knowledge. Detailed C2 
chooses the most effective COA by trying to develop a complete picture of the 


battlespace. 


The commander’s C2 philosophy affects his choice of actions. A mission C2 
commander may decide to take actions to accomplish his objective in the face of an 
incomplete or uncertain picture of the battlespace. Given the same situation, a detailed 
C2 commander may choose to request guidance from his superior or continue to gather 
information. _SSIM CODE’s C2 philosophy is considered in the decide phase of the 
commander’s OODA loop. 


OF AN INDIVIDUAL COMMANDER’S DECISION-MAKING 


A commander’s individual characteristics are considered in SSIM-CODE’s 
decision-making process. The commander is modeled as an entity with properties (Java 
object attributes) including a command style (conservative vs. aggressive), a C2 


philosophy (mission vs. detailed), and an experience level (high or low). 
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Command style influences the likelihood of certain actions in a given situation. 
For example, an aggressive commander applying mission C2 is more likely to attack in 
an environment that makes an attack difficult. A conservative, detailed C2 commander 


may elect to bypass the enemy in a similar situation. 


The commander’s C2 philosophy determines the application of mission or 
detailed C2. A commander with a mission C2 philosophy is more likely to make a 
decision without requiring further direction from a higher command or increased 
certainty. Probabilities associated with specific actions in SSIM CODE are determined 


by the commander entity's C2 philosophy and C2 style. 


An inexperienced commander takes more time to process incoming information, 
develop his orientation, and reach a decision. In SSIM CODE, the commander’s 
experience level is used to determine time delays in the phases of the commander’s 
OODA loop. In the SSIM CODE evaluation, discussed in Chapter V, all combinations of 
commander attributes are examined. Chapter VII describes potential sources for 
populating these attributes in practice. 


1. Commander’s Intent 


Commanders deliver a commander’s intent to their subordinates as a means to 
communicate the key elements of a mission. Marine Corps Doctrinal Publication 
(MCDP)-1 Warfighting describes commander’s intent as a tool for subordinates to 
“understand the larger context of their actions.” Tactical commanders rely on their 
superior’s commander’s intent to focus their decision-making and assess the effectiveness 


of their decisions. 
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A commander’s intent can include the commander’s focus, concerns, CCIRs and 
desired end-state. The most important element of the commander’s intent is typically the 
desired end-state (Posadas, 2000). The effect of time as a critical element of the mission 
is typically captured in the commander’s intent. In SSIM CODE, a simplified 
representation of a commander’s intent is included by the desired end-state. The end- 
state in this model is comprised of a quantitative objective description and an end-state 
event. For example, the desired end-state may consist of achieving a 1:3 friendly to 


enemy force ratio within two hours of detecting enemy forces in a specific area. 


Expanding the SSIM CODE model could develop a more robust commander’s 
intent. However, the purpose for the commander’s intent in SSIM CODE is to define a 
means for evaluating the effectiveness of the tactical commander’s decisions. A 
simplified, quantifiable commander’s intent achieves this purpose. 


2. Decision Factors 


The commander entity in SSIM CODE is linked to an SA module in Combat 
XXI. The SA module monitors information throughout the combat simulation, maintains 
a collection of perceived facts, starts the commander’s decision cycle when a decision is 


required, and implements actions that result from the commander’s decisions. 


Decision factors, influenced by state variables in the combat simulation, are 
updated in the commander’s observe decision phase. Decision factors are binary discrete 
random variables computed as functions of varying states in the combat simulation. 
Decision factors are aggregated elements that influence tactical decision-making. While 
decision factors have discrete states, commander entities in SSIM CODE do not have 


direct access to the discrete states. Commander entities are provided probabilistic 
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estimates of decision factor states (uncertain information). This concept is developed in 


more detail later in this chapter. 


Key decision factors are described by MCDP-6 Command and Control. When 
describing the observe phase of the OODA loop, MCDP-6 states: “...we take in 
information about our own status, our surroundings and our enemy.” (U.S. Marine Corps, 


1996) 


Examples of decision factor states include: whether the condition of the 
commander’s own forces is positive or negative, the favorable or unfavorable state of the 
environment (relative to a specific action), and the weak or strong state of enemy forces. 
Based on this guidance, SSIM CODE captures the essential elements of military 


judgment with three-decision-factors: own forces, environment, and enemy forces. 


The model could employ an abstract n-factor design. However, according to 
MCDP-1-3 Tactics, a tactical commander develops his understanding of a situation by 
specifically considering METT-T (U.S. Marine Corps, 1997). The key elements in 
METT-T are enemy forces, the environment, and friendly forces, according to MCDP-6 


Command and Control (U.S. Marine Corps, 1996). 


In SSIM CODE, mission and time available are elements of the higher 
commander’s intent. The decision factors represent the commander’s consideration of 
his troops (own forces), terrain and weather (environment), and the enemy (enemy 
forces). Thus, the five elements of METT-T are represented in SSIM CODE. Table 1 


lists the states for the three binary decision factors. (A discussion of multinomial 


29 


decision factors is included later in this chapter.) These states are similar to those applied 


by MCCDC in the Bayesian network decision-making model (Stephens, 1998). 


























Decision Factor Description States 
B Condition of Own Forces P = positive 
(Blue) N = negative 
R Condition of Enemy Forces S = strong 
(Red) W = weak 
E Environment State Relative to F = favorable 
(Environment) Own Mission U = unfavorable 














Table 1. Decision Factor States 


Each decision factor’s state can be determined by observations on related state 
variables from within the combat simulation. State variables from within Combat XXI 
are indicators for decision factors in SSIM CODE. For example, the condition of the 
commander’s subordinates (own forces factor) can be determined by measuring the 
degree to which the forces are engaged with the enemy (represented by the Combat XXI 
variable platformEngagementFactor (TRAC-WSMR, 2001)) and the amount of damage 
incurred (denoted by the variable platformDamageFactor in Combat XXI (TRAC- 
WSMR, 2001)). Additional indicators of the state of own forces may be considered; 
however, a balance is sought between the number of state variables required to determine 
a decision factor state and an adequate representation of the decision factor’s state. 


D. CONDITIONAL PROBABILITY MODEL 
1. Detailed Model 


The decision-making process can be modeled in detail with conditional 
probabilities. First, the elements that influence the commander’s decision are 
determined. The commander’s experience level (X), C2 style (Y), and C2 philosophy 
(Z) influence his decision-making. His perception of the higher commander’s intent (C) 
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also influences his decisions. The actual situation (S) is equivalent to the combined state 
of the three decision factors in the Bayesian model. The commander’s estimate or 
perception of the situation (I) is based on reports on the actual situation (S). Figure 10 is 
an influence diagram (Marshall, 1995) that represents the probabilistic dependencies 


between the elements of decision-making. 
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Figure 10. Influence Diagram of a Decision-Making Process 


The influence diagram is ordered in time from left to right. The commander’s 
attributes are determined (X, Y and Z), and then the commander develops a perception of 
the higher commander’s intent (C). Next, the commander develops an estimate of the 


situation (I) and makes his decision (D). The consequence of a decision is a result (R). 


The directed arcs denote possible conditional dependence. The absence of an arc 
between two nodes indicates possible conditional independence (Marshall and Oliver, 


1995). The commander’s estimate of the situation (I) and perception of his mission (C) 
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depend on his experience level (X). His decision (D) depends on his C2 style, C2 
philosophy, situation estimate and his mission perception (Y, Z, I, and C). However, the 
commander’s decision is conditionally independent of the actual situation (S), given the 
estimate of the situation (1). The result of the commander’s decision depends on the 
commander’s decision (D) and the actual situation (S), but given these two factors, the 
result is conditionally independent of the other elements. The following distributions 


(data) are required to solve the conditional probability model: 





Marginal Distributions 
P{S=s} 
P{X=x} 
P{Y=y} 
P{Z=z} 


Conditional Distributions 
P{C=c | X=x, Y=y} 

Pil=t |\X=x%7=z2 S=s} 
PiDed | Tai, Cae, Yay, Z=2} 
P{R=r | S=s, D=d} 


Joint Distributions 

P/ X=x, Y=y} 

PYXa=i, 77 S=s)} 
Pil=t:C=¢ *=y, Z=2 7 
P{S=s, D=d} 

















Table 2. Required Data for Conditional Probability 
Model 


For the purposes of this thesis, the commander’s decision probabilities in SSIM 
CODE (P{D=d | I=i, C=c, Y=y, Z=z }) are based on expert judgment for the specific 
evaluation scenario described in Chapter V. A discussion of potential means for 


populating such a conditional probability distributions is included in Chapter VII. 
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According to this model, the commander’s attributes (X, Y and Z) are the most 
influential elements in his decision-making. This is illustrated when solving for the 
marginal distribution of the commander’s decision: 


P{D=d} = P{D=d | I=i, C=c, Y=y, Z=z } 
‘P{I=i} P{C=c} P{Y=y} P{Z=2} 


P{D=da}= P[D=d \J=1, C=¢, Y=y,:Z=z24 
‘P{T=i | X=x, Z=z, S=s}-P{X=x} P{Z=z} P{S=s} 
* P§C=c | X=x, Y=y} P{X=x} P{V=y} P{V=y} -P{Z=z} 














P{D=d}= P{D=d | I=i, C=c, Y=y, Z=z } ‘P{I=i | X=x, Z=z, S=s}P{S=s} 
- P{C=c | X=x, Y=y}P{X=x}? P{Y=y}? -P{Z=z}" 


This value of P{D=d} is expressed in terms of required data. The marginal 
probabilities of the commander’s attributes (P{X=x}, P{Y=y}, P{Z=z}) appear as 
squared terms in the solution for a decision outcome (R). These terms have the most 
influence on P{D=d}. Thus, the commander’s attributes are expected to be the most 


influential elements of his decision-making. 


Solving for the probabilistic result yields: 
P{R=r} = P{R=r | S=s, D=d}-P{S=s}-P{D=d} 
P{R=r} = P{R=r | S=s, D=d}-P{D=d | I=i, C=c, Y=y, Z=z } 


‘P{l=i | X=x, Z=z, S=s}-P{C=c | X=x, Y=y} 
‘P{X=x}? -P{Y=y}? P{Z=z}’ -P{S=s}? 








The resulting outcome is influenced most by the actual situation (S) and the 
commander’s attributes (X, Y, and Z). This analytical model is informative in evaluating 


the probabilistic relationships between decision-making elements. 
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The quality of the commander’s decision-making process could be analyzed with 
the conditional probability model by comparing the decision result with a desired 
outcome. However, the conditional probability model requires a substantial amount of 
data (listed in Table 2) in the form of probability distributions. For example, the 
marginal distribution that a commander is aggressive, the conditional probability of a 
commander’s decision (given information, commander’s intent, C2 style, and C2 
philosophy), and the joint probability of commander experience and command style are 
among the required data to attain an outcome. Extensive prior probabilities would be 
necessary for a single calculation. 


2. Simplified Model 


The SSIM CODE model is a simplified version of the conditional probability 
model. The simplification is required to reduce the quantity of data used to determine 
decision outcomes and to decrease computational complexity. To simplify the model for 
simulation, a given set of attributes are assumed for each commander. SSIM CODE 
assumes the experience level, C2 philosophy and C2 style of a commander are known or 
can be estimated. Commander attributes are deterministic parameters provided to SSIM 


CODE. 


The commander’s perception of his mission, or higher commander’s intent, is 
defined as a set of rules (based on expert tactical judgment) in SSIM CODE. This 
perception varies with the commander's individual attributes. Thus, the decision outcome 


is influenced by the commander’s attributes. 


The probability of a commander’s decision outcome, given his attributes and his 


estimate of the situation (decision factors), remain required data for SSIM CODE. State 
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variables in the Combat XXI simulation define the actual situation at any specific time. 
The estimated situation is a probabilistic input to the commander entity in SSIM CODE. 
Reports on decision factors estimate the situation and represent the degree of uncertainty. 
E. ABSORBING MARKOV CHAIN MODEL 

An absorbing Markov chain (Ross, 1997) model can determine decision outcomes 
based on the simplified model. Modeling the commander’s decision-making process 
with an absorbing Markov chain results in a probabilistic decision outcome that reflects 
the variability associated with human decision-making and represents the uncertainty 
inherent to the commander’s estimate of the situation. 

1. State Space 

A discrete time Markov chain can describe the OODA loop process. Each phase 
in the OODA loop is a discrete time step. The decision factors are represented by the 
variables E (environment), R (enemy forces), and B (own forces), in accordance with 


METT-T. 


The states in the Markov chain model correspond to decision factor states. For 
example, F is the state where the environment decision factor is favorable. U represents 
an unfavorable environment decision factor. The state F,S describes a favorable 
environment and a strong enemy. Three factor states describe a complete perception of 
the battlespace, such as F,S,P: favorable environment factor, strong enemy forces factor, 
and positive own forces factor. There are eight such states (the number of states 
increases exponentially with the number of factor levels). The decision outcomes (e.g., 


attack or bypass) are absorbing states. 
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2. Markov Chain Calculations 


To determine the stochastic outcome of the decision-making process, decision 
factor estimates and commander’s attributes are combined in an absorbing Markov chain 
model. A transition matrix is populated based on probabilities associated with each 
decision factor. The probabilities in the transition matrix are drawn from decision factor 
reports (probabilistic states) provided to the commander and from the commander’s 
decision outcome conditional probabilities. For example the commander would receive 
reports that detail P{R=r}, P{B=b}, and P{E=e}. An individual commander’s attributes 


include conditional probabilities for each possible combined state such as: 


P{D=d | R=r, B=b, E=e}. 


The long-run probability matrix (Ross, 1997) is then calculated. Finally, the 
probability associated with each decision outcome is retrieved from the long-run 


probability matrix. 


Figure 11 is an example of a transition diagram for a decision to attack or bypass 
an enemy force. Reports to the commander describe observations on the environment, 
enemy forces and his own forces. The reports detail the probability that a decision factor 
takes on a specific state value (e.g., P{E=favorable}=.75). The transition probability 
from a combined state (e.g., to E=favorable and R=strong and B=positive) to a decision 


outcome is determined from the commander’s C2 style. 
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Figure 11. Decision-Making Transition Diagram 


From this transition diagram, a transition matrix, P, is constructed: 


0. Attack 1.Bypass 2 Decide 


0. Attack { 
1. Bypass 0 
2. Decide 0 
3. F,S,P Peg 
4. FS.N Pag 
P= 5 FWP Psp 
6. FN Pep 
7. USP Pag 
8. USN Pea 
9. UWP Pag 
10. U,W,N Poo 





0 








So 2c 0,e'0 Co So oe « 


3.F,5P 4.7,5N 5.F WP 6.FWN 7USP 8 USN 9 UWP 10.U, WN 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
Pog Pog Pog Pog Poy Pog Pog Pata 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 





























Probabilities P23 through P2190 are derived from the decision factor reports, 


Probabilities P30, P3,1,..., Pio, Pio, are the commander’s decision-making conditional 


probabilities for each combined state. 


States 0 and 1 are absorbing states; states 2 


through 10 are transition states. Every transition state has access to an absorbing state. 
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Future states are conditionally independent of previous states, given the current state. 


Therefore, the conditions for a Markov chain are met. (Ross, 1997) 


For an absorbing Markov chain, the probability of ever reaching state j given that 
the decision process starts in state i (f;) is given by: fj, = [I - Q] # Rj (Ross, 1997). This 
result is the decision outcome probability: f;; = P{D=d/. The matrix Q holds transient- 
to-transient transition probabilities, R holds transient-to-absorbing transition probabilities 
and / is the identity matrix. 


2.Decide 3.F,5P 4.F,5N 5.F,WOP 6F,WN 7.U5,P 8.U,5N 9. U,WP  10.U, WN 


2. Decide | 0 P23 Pag Pag Pag Paz Pag Pag Pato 
3. F,S,P | 0 0 0 0 0 0 0 0 0 
4. F.S,N I) 0 0 0 0 0 0 0 0 0 
5. FWP | 0 0 0 0 0 0 0 0 0 
QO = 6. F,WN | 0 0 0 0 0 0 0 0 0 
7. USP | 0 0 0 0 0 0 0 0 0 
8. USN | 0 0 0 0 0 0 0 0 0 
9. UWP | 0 0 0 0 0 0 0 0 0 
10. UAWN | 0 0 0 0 0 0 0 0 0 
0. Attack = 1.Bypass_ 
2. Decide | 0 0 
3. FSP | Psa P34 
4 FSN | Pao Pay 
pes 5. FWP | Psa Psa 
6. F,VWAN | Peo Pea 
7. USP | Pro Pa 
8. USN | Peo Psq 
9. UWP | Poo Pai 
10. U,W,N | Proo P4o1 


3. Outcome Probabilities 

The absorbing Markov chain long-run probability matrix yields a probability that 
a decision outcome (absorbing state) is reached. A decision with two possible outcomes 
(e.g., attack or bypass) can be described as a Bernoulli trial (a special case of a Binomial 
trial, e.g., success=attack, failure=bypass). The probability of success is used as the 


Bernoulli distribution parameter. A uniform (0,1) random number is then generated and 
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compared with the probability of success (e.g., probability of attack). If the uniform 
random draw is less than the probability of success, the decision outcome is set to success 
(e.g., attack). Otherwise, the decision outcome is set to failure (e.g., bypass). (Law and 


Kelton, 2000) 


For decisions with more than one outcome (multi-nominal trials), the Markov 
chain model would yield a probability associated with each outcome. For example, for 
three outcomes, A, B, and C, the probabilities can be denoted as: P{A} = p;, P{B} = pz, 


and P{C} = p3, where p7+p2+p3= /. 


To determine the outcome chosen by the commander, a uniform (0,1) random 
number, U, is then generated and compared with the probabilities. For p;< p2 < p3, the 
outcome would be A if 0 < U<p), Bif pr < U < (pi+pz2), and C if, (pitp2) < U < J. 


This procedure can be generalized to a decision with n outcomes: 


k-1 k 
Apply outcome k, if DS pi<Us ~ Di. 


i=l i=l 


4. Computational Complexity of the Markov Chain Model 


The absorbing Markov chain model effectively uses decision factor states and 
commander attributes to produce probabilistic decision outcomes. However, the matrix 
operations required for each decision outcome result in a large computational complexity. 
When the transition matrix, P, has dimensions nxn, calculating an outcome probability 
with the Markov chain decision-making model involves on the order of n operations 
(multiplications and additions). Using the notation in Ahuja, Magnanti and Orlin (1993), 


this model has a complexity of O( n°). 


39 


Each decision’s outcome probability is determined by f= [J - Q/ Rj as 
described in the Outcome Probabilities section. With an nxn transition matrix, the 
complexity of (J — Q) is O(n). Inverting the resulting matrix has complexity between 
O(n** ) and O(n’), depending on the algorithm used (Ehrling, 1999). Thus, a single 


outcome probability calculation involves O( n°) computational complexity. 


The number of binary decision factors and the number of decision outcomes 
determine the transition matrix dimensions (nxn) and the computational complexity. For 
an m-outcome decision with k binary decision factors, n = 2° +m + 1. In terms of 


decision factors, the complexity of the Markov chain decision-making model is o(2"*). 


Because Combat XXI is a high-resolution combat simulation, it is required to 
continuously generate a large number of computational results. Adding unnecessary 
computational complexity to the simulation is an undesirable effect of the Markov chain 
model. A model with similar functionality, but reduced complexity would be more 
appropriate for a high-resolution combat simulation. A Bayesian network model 
provides such features. 


F. BAYESIAN NETWORK MODEL 


A Bayesian network model yields identical probabilistic outcomes as the 
absorbing Markov chain model with less computational intensity. The three decision 
factors in SSIM CODE (environment, enemy forces, and own forces) are applied to the 


Bayesian network model to determine an outcome: the commander’s decision. 


The decision outcome is probabilistically dependent on the decision factors. The 


decision factors (random variables) make up a joint probability distribution for the 
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commander’s decision outcome (Stephens, 1998). Figure 12 shows a sketch of a 


Bayesian network with three decision factors. 





Outcome 


Decision Factors 
—Blue Cdr’s Decision 


» 2-State Discrete Random Variables 
> Functions of State Variables 


> 3 Decision Factors 
* Own (Blue) Force Condition 
¢ Enemy (Red) Force Condition 
* Environment 


B 
Condition of R 
Blue Forces Condikom of 
Red Forces 
BD 
Biue Decision 


Figure 12. Bayesian Decision-Making Network (After Stephens, 1998) 











1. Bayesian Network Model Decision Outcomes 


A decision outcome probability from this Bayesian network is determined by: 


P{BD=d} = P{BD=d\|B=b, E=e, R=r}-P{ B=b}-P{ E=e}-P{ R=r} 
+ P{BD=d|B=b, E=e, R=r}-P{B=b}-P{ E=e}-P{R=r} 
+ P{BD=d|B=b, E=e, R=r}-P{ B=b}-P{ E=e}-P{ R=r} 
+ P{/BD=d|B=b, E=e, R=r}-P{ B=b}-P{E=e}-P{R=r} 
+ P{BD=d|B=b, E=e, R=r}-P{B=b}-P{ E=e}-P{R=r} 
+ P{BD=d|B=b, E=e, R=r}-P{ B=b}-P{ E=e}-P{R=r} 
+ P{/BD=d\|B=b, E=e, R=r}-P{B=b}-P{ E=e}-P{R=r} 
+P{BD=d|B=b, E=e, R=r}-P{ B=b}-P{ E=e}-P{ R=r} 


The terms on the right side of the expression are data required by SSIM CODE. 


This is the same set of required data used in the Markov chain calculation. The 
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commander’s decision-making conditional probability is P{BD=d | B=b, E=e, R=r }. 
Reports to the commander define P{B=b}, P{E=e}, and P{R=r}. The probabilistic 


decision outcome is identical in value to the result from the Markov chain model. 


The computational complexity for the Bayesian Network calculation is O(2"), for 
k decision factors. This is a significant (exponential) reduction in computational 
complexity compared to O(2**) for the Markov chain model. So, for the same data 
requirement, the Bayesian Network model saves on computational effort. 

2. Stochastic Decision-Making 

The decision factor states and the commander’s attributes determine the Bayesian 
network’s underlying joint probabilities. For example, the outcome of a specific 
commander’s decision to attack has several probabilistic outcomes depending on decision 
factor states: 

P{Attack | B=positive, E=favorable, R=weak} = .95 

P{Attack | B=positive, E=favorable, R=strong} = .50 


P{Attack | B=negative, E= unfavorable, R=weak} = .30 
P{Attack | B=negative, E=unfavorable, R=strong} = .15 


The commander’s attributes determine the probability of a specific decision 
outcome. The probability that a commander makes a certain decision, given decision 
factor observations, varies with the individual qualities of the commander. For example: 


If C2 Style=aggressive, 
Then P{Attack | B=negative, E=unfavorable, R=weak} = .40 


However, if C2 Style=conservative, 
Then P{Attack | B=negative, E=unfavorable, R=weak} = .20 
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The probabilities assigned to decision outcomes (for each set of commander 
attributes) would be provided to SSIM CODE as data in the same manner as the 


commander attributes. Scripted probabilities were used to test SSIM CODE. 


Figure 12 shows a Bayesian network in which a commander’s decision depended 
on direct observations of EF, R, and B. In reality, commanders may not have direct access 
to this information. For example, a company commander does not know the actual state 
of enemy forces (i.e., he cannot readily observe the enemy directly and determine the true 


state of enemy forces). He bases his decisions on intelligence estimates. 


In practice, commanders make decisions based on reported estimates—not on 
perfect information. To model this concept, additional nodes are introduced to the 


Bayesian network. These nodes represent reports on decision factor states. 


Three sets of nodes are now depicted in the Bayesian network: the commander’s 
decision, reports and decision factors. The lack of perfect information in tactical 
decision-making is captured in the relationship between the three sets of nodes. The 
decision outcome is probabilistically dependent on report states and independent of 
decision factor states. Figure 13 depicts the probabilistic dependencies of a model with 


imperfect information. 


43 








Reports 
> Represent Imperfect Information 
>» Contain Uncertainty 


E 
Environment 
R 


we Condition of 
Red Forces 
R, B 
Report 1 Condition of 
Blue Forces 


BD 
Blue Decision 





Figure 13. Bayesian Network with Reports (After Stephens, 1998) 


The report nodes in the expanded network represent the uncertainty inherent to 
the commander’s information. Based on the Bayesian network in Figure 13, the 


commander’s decision is conditionally independent of FE, R and B, given R;, R2 and R3. 


In SSIM CODE, the commander entity does not make direct observations of the 
decision factors. For example, while the environment has a deterministic state (favorable 
or unfavorable), the commander only has access to an estimate of that state P{/E=f} or 
P{E=u}. He may receive a report estimating the probability of a favorable environment 
at 85%. The commander may be misinformed and has to weigh the uncertainty in a 


decision factor report. 


For example, given perfect information, a specific commander may attack with 


95% probability if the combined state is: B=positive, E=favorable, R=weak. But since 
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his information is imperfect, the commander must decide based on uncertain reports: 


P{B=positive}=0.90, P{E=favorable}=0.60, P{R=weak}=0.55. 


After weighing the uncertainty, the commander will attack with a 68% 
probability. The difference between the 95% likelihood to attack and the 68% likelihood 
to attack is a result of the disparity between reality and the commander’s perception or 


orientation. (Stephens, 1998) 


The use of this Bayesian network model to determine decision outcomes 
introduces decision variability. Given the same information, the commander will not 
always reach the same decision. This decision model also accounts for uncertainty. The 


commander bases his decisions on inexact estimates of decision factors. 
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IV. MODEL DESCRIPTION 


A. MODEL STRUCTURE 

SSIM CODE is applied in Combat XX] as a platform functionality module. This 
module includes a platform object containing attributes such as a location (grid 
coordinate) and type (e.g., MIA1). A platform has the potential to move, search, 
communicate, etc. (enabled by adding appropriate functionality). The primary element of 


SSIM CODE is the commander entity. The commander entity has an SA module. 


In Combat XXI, an SA module maintains facts and executes actions. SSIM 
CODE is capable of information exchange with Combat XXI by interfacing with the SA 
module. As a functionality module for Combat XXI, SSIM CODE requires input from 
various elements of the simulation to execute the commander’s decision cycle and to 
implement decisions. Through the SA module, SSIM CODE monitors changes in state 


variables, monitors SimEvents, and has access to the current battlespace. 


The facts/actions expert system in the SA module updates facts applicable to the 
commander’s decision cycle. This expert system applies rules (e.g., representations of 
doctrinal tactics) to translate a commander’s decision to a set of primitive commands 
(engage, move, search, etc.). Figure 14 depicts the structure of the interactions between 
SSIM CODE and Combat XXI. This figure delineates which components are parts of 


SSIM CODE and which exist in Combat XXI. 
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Figure 14. Model Structure 
The decision cycle is an attribute of the commander entity. Decision outcomes are 
communicated to the Combat XXI simulation through the commander’s SA module. 
Observations on simulation events and properties are communicated to the commander 
through the SA module. 
B. MODEL OVERVIEW 
The SSIM CODE model is centered on the commander entity. The SSIM CODE 
commander entity possesses an SA module, a C2 style, a C2 philosophy, an experience 
level, and a set of decision cycles (OODA loops). Three types of decision cycles are used 
to test SSIM CODE—one for each decision type (engage, search, move). While several 
decision cycles may be active in one commander entity, only one of each specific OODA 
loop type is active at a given time. For example, only one move decision can be in 


progress, but one engage and one search decision could be active at the same time. 


1. The SSIM CODE OODA Loop 


A SimEvent in SSIM CODE initiates each phase of the commander’s OODA 


loop. The commander’s SA module determines when a decision is required. OODA 
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loops are activated periodically by the commander or by external simulation events and 
property changes. The time required to complete each phase of the OODA loop is 
determined by the commander’s experience level. 

Each OODA loop phase has a time delay. These delays are modeled with 
exponential distributions. The exponential distribution is memory-less and it “...1s 


frequently used as a model for the distribution of times between the occurrence of 


successive events.” (Devore, 1995). 


CCIRs are key questions that the commander wants resolved to focus his decision 
process (U.S. Marine Corps, 1996). These CCIRs determine which events and property 
changes trigger OODA loops by requiring decisions. Decisions required while the 
OODA loop of the same type is active are placed in a decision queue. The commander 


addresses pending decisions upon completion of his current OODA loop. 


In the observation phase, reports, and commands from higher headquarters are 
received. If the commander entity employs a detailed C2 philosophy, additional 
information is requested to improve the accuracy of the observation. This request for 
more information increases the duration of the observation phase. Commander entities 
with mission C2 philosophy accept the accuracy of the reports and continue with the 
decision cycle. In the orientation phase, a combined state is determined based on 
updated decision factors. The decide phase applies the decision factor observations and 
commander’s attributes to the stochastic decision process to obtain an outcome for each 
decision. The Bayesian network model is implemented in this phase. The resulting set of 


decisions constitutes the commander’s COA. 
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The action phase of the commander’s decision cycle requires interaction with 
other entities in the simulation. The facts/actions expert system accomplishes this 
interaction by issuing orders, from the commander, to be executed by subordinate 
entities. The action phase involves translating the commander’s decisions into a set of 
actions that represent the commander’s decision (move, engage, search, etc.). This output 
is then communicated to the appropriate entities through the SA module. 


2. The SSIM CODE Decision-Making Process 


SSIM CODE includes report objects and decision objects. Reports provide 
information on decision factors with a degree of uncertainty. Decisions are developed as 
the OODA loop progresses. Figure 15 is an event graph (Buss, 2000) overview of the 
SSIM CODE model. This figure illustrates the event sequence in a commander’s 


decision cycle. 
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Figure 15. SSIM CODE Event Graph Overview 
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First, a SimEvent from within Combat XXI triggers a CCIR by the change of a 
significant property (state variable) or the occurrence of a key event that answers a CCIR. 
The commander entity in SSIM CODE monitors these changes through its SA module. 
When a decision is required, the appropriate type of OODA loop is started. Reports on 
decision factors are received and a perception of the current situation is developed 
through a combined decision factor. The Bayesian model is implemented to determine a 
decision outcome. Next, actions are taken to implement the decision. Facts are then 


updated in the SA module and any subsequent decisions are scheduled. 


In Chapter I, desirable attributes of a tactical decision-making model were listed 
as: a representation of the commander’s decision-making process, an emphasis on his 
perception, a portrayal of uncertainty and chance, and a decision cycle structure. SSIM 
CODE uses the OODA loop decision cycle structure to represent the tactical 
commander’s decision cycle. The commander’s perception is emphasized in the orient 
phase of the OODA loop. In this phase, SSIM CODE employs decision factors based on 
C2 doctrine and develops the commander’s perception based on reports that include 
uncertainty. A Bayesian network determines the decisions generated by the commander 


entity. These probabilistic decision outcomes characterize chance. 


SSIM CODE addresses each of the three information-processing styles. Reports 
on decision factors are received by the commander entity in a set order as in directed 
processing. CCIRs are used to initiate decisions, as in triggered information-processing. 
Allowing decisions to induce subsequent decisions and further inquiries about facts as the 


commander entity develops a COA represents inquiry-based information-processing. 
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Chapter II added the commander’s C2-related characteristics and the 
commander’s evolving awareness as key components in simulating C2 decision-making. 
SSIM CODE depicts the commander’s experience level, C2 style and C2 philosophy and 
employs these attributes as influences on decision-making. The commander’s dynamic 
SA is portrayed through the link between SSIM CODE and the Combat XXI SA module. 


C. MODEL ASSUMPTIONS 


General assumptions were made in the development of the SSIM CODE model. 
The model can be expanded for additional robustness. However, based on these general 
assumptions, the SSIM CODE model with three decision factors, three commander 
attributes, and three decision types provides suitable insight to evaluate its performance 
as a tactical decision-cycle model. The general assumptions include: 


® The three decision factors (environment, own forces, enemy forces) provide 
the commander with an adequate perception of the battlespace. 


® The scripted commander attributes provide a reasonable depiction of a 
tactical commander. 


" Three decision types (engage, search, move) with two outcomes each 
provide sufficiently robust COAs. 


Assumption | states that the commander’s observation of the battlespace can be 
derived by the elements of METT-T. The mission and time requirements are given in the 
higher commander’s intent. To make a decision, the commander must consider the 
terrain (environment factor) the enemy (enemy forces factor) and his own troops (own 
forces factor). These three decision factors represent a thorough observation to the 


battlespace. 
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Assumption 2 maintains that the attributes chosen to describe a commander (C2 
philosophy, C2 style, and experience level) adequately capture the characteristics that 


affect a commander’s decision-making. 


The third assumption states that a tactical commander’s COA may be described as 
a series of decisions on whether or not to search, engage or move. The essence of a 


tactical course of action is assumed to be portrayed by these three actions. 
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V. MODEL EVALUATION 


A. TEST SCENARIO 


SSIM CODE was tested as a stand-alone entity. The evaluation was loosely 
coupled with Combat XXI. The test scenario was executed from within Combat XXI. 
SSIM CODE interacted with Combat XXI through the SA module and Simkit event 
scheduling. The decision factors, commander attributes, and enemy COAs were scripted 
to support analysis using a factorial design. A general scenario description is developed 


in this section. Specific scenario parameters are listed in Appendix A. 


The scenario involves an infantry company in the defense (Blue defending against 
Red). The SSIM CODE test scenario models a company commander’s decision cycle. In 
the test scenario, only the Blue forces apply the SSIM CODE decision cycle model. Red 
forces employ scripted actions. The company commander considers three decision 
factors in his decision-making: the state of the environment, the state of enemy forces, 
and the state of his own forces. His COAs consist of decisions to move, search and 


engage. 


The company commander entity makes tactical decisions to accomplish an 
assigned mission. The company commander’s objective is drawn from a battalion 
commander’s intent. The company commander’s decisions are communicated to three 


subordinate platoons. 


The Blue company is situated in an assembly area while preparing to establish a 


defense. The Blue company commander's mission is to block any of three avenues of 
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approach (AAs) used by the enemy. His forces have three battle positions (BPs) 
available to establish a defense. Each of the BPs is associated with a targeted area of 
interest (TAI) and an AA. TAIs are engagement areas. The company commander is 
tasked to allocate forces to the appropriate BPs to best defend against an advancing 
enemy. The battalion commander’s intent specifies the requirement to attain a 1:3 (Blue 
to Red) force ratio at each BP/TAI pairing by a certain time after the enemy reaches the 
TAI(s). The battalion commander has also directed the company commander to engage 


enemy forces once they entered a TAI. Figure 16 illustrates the test scenario. 
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Figure 16. SSIM CODE Test Scenario 


The company commander receives reports on the enemy from two named areas of 
interest (NAIs). Observations of enemy actions at each NAI indicate whether the enemy 
intends to use AA 1, 2, 3 or a combination. Terrain restricts enemy movement to the 


three AAs. Only one NAI may be monitored at any time. The sensors used to monitor 
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the NAIs have a fixed detection error. (This detection error is assumed to be the 
combination of several random effects and is thus modeled with a Normal distribution 
(Devore, 1995)). Reconnaissance teams constantly monitor TAIs. TAI observations are 


also subject to detection error. 


Enemy forces at each TAI can only be engaged from the corresponding BP. Once 
the company commander directs forces at a specific BP to engage, those forces must 
remain in place. The company commander moves and engages with platoon size 


elements (3 units). The enemy moves in companies (3 units). 


To meet the battalion commander’s intent, the company commander must decide 
whether to search in NAI 1 or 2, whether to move to BP 1, 2 or 3 and whether to engage 
the enemy at TAI 1, 2 or 3. The company commander must attain the specified force 


ratio by the specified time. 


The evaluation includes analyzing the effect of time on tactical decision-making 
in SSIM CODE. The battalion commander has allocated close air support (CAS) assets 
to hold the Red forces in the TAIs for a specific period of time. The CAS period starts 
when the last Red company reaches a TAI and ends a specified time after the last Red 
Company arrives at a TAI. The Blue company commander must attain the specified 
force ratio by the end of this CAS period. Several parameterized CAS period times were 


used as described in Chapter VI 


The test scenario is an initial evaluation of SSIM CODE and includes several 
simplifications. Weapon systems and attrition are not represented in this scenario. The 


scenario simply serves to evaluate the Blue commander’s decisions to allocate forces. 
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Once the command to engage is issued to a Blue platoon, that remains in place. Force 
ratios are determined at the meeting point of the two forces without adjusting for attrition. 
The CAS period serves to hold Red forces in place at the TAIs, but does not diminish 


their force strength. 


Marine Corps doctrinal C2 is applied in the scenario. Doctrinal planning and 
decision-making tools such as NAIs and TAIs are used. The company commander entity 
is issued a battalion commander’s intent and a mission. A set of rules is developed to 
represent doctrine, the battalion commander’s intent and the company commander’s 
CCIRs. Responses to enemy actions (decisions) are determined by the commander's 
decision-making. The simulation is stopped after all Red forces reach a TAI and the 
Blue CAS period expires. The end-state BFR is used as a measure of Blue's success. 


B. DECISION RULES 


The commander entity applies a set of decision rules to attain the mission goal 
(the assigned force ratio). These rules are invoked according to the decision outcomes. 
One set of rules details the company commander’s CCIRs. Three other sets of rules 
describe the commander’s actions according to the outcomes for search decisions, 
movement decision and engagement decision. Appendix B details the decision rules used 
in the test scenario. 


1. CCIR Rules 


The Blue company commander considers enemy detections as critical 
information. Each time Red forces are detected at an NAI or TAI, the Blue commander 
would like to be informed. SSIM CODE accomplishes this by establishing an event key 


for each type of enemy detection. When an enemy detection SimEvent takes place, the 


58 


commander entity’s SA module updates the commander’s set of facts and triggers a 


CCIR action. This is triggered information-processing in SSIM CODE. 


The CCIR action is invoked to determine what further action is required and 
results in the scheduling of a decision. This is an example of SSIM CODE’s inquiry- 
based information-processing. Based on the detection event that triggered a CCIR action, 
either a search decision, a movement decision, or an engagement decision is scheduled. 


2. Search Rules 


In the act phase of a search decision, the commander entity applies the decision 
outcome through a set of search rules. The commander entity's OODA loop sets a 
decision outcome, the SA module updates the commander’s facts, and the search rules 
are invoked to carry out the decision action. Then, a choice is made: search in NAI 1 or 
in NAT 2. 


3. Movement Rules 


A movement decision invokes the movement rules through the commander 
entity's SA module. The decision outcome determines whether Blue forces will be 
moved or not. The updated enemy detections are considered and Blue platoons are 
ordered to a BP according to the interpretation of the Red force movement. 


4. Engagement Rules 


The decide phase of a commander’s engagement decision sets the decision 
outcome. The act phase invokes the engagement rules through the commander’s SA 
module. Detections of enemy forces at the TAIs are considered by the engagement 


rules. Aggressive commanders may engage with up to all three platoons. Conservative 
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commanders engage with only one platoon at a time. Once a platoon engages the enemy, 
it is unavailable for further movement. 


Cc. SCRIPTED COMMANDER ATTRIBUTES 


Commander attributes (experience level, C2 philosophy, and C2 style) were set at 
a specific level for each experimental run in the SSIM CODE evaluation. These 
attributes remain constant throughout the entire run. The arrangement of attribute levels 


per run is described in the factorial design section. 


Commander decision probabilities used in the Bayesian network calculations 
depend on the decision type, the combined state (decision factors) and the commander’s 
C2 style. Probabilities, for each of the eight possible combined states, were set according 
to the commander’s C2 style. Appendix A lists the probabilities used in the test scenario 
and shows decision probabilities for each decision type and C2 style across the eight 
possible combined states. These probabilities were chosen by expert judgment and are 
intended to reflect the differing nature between aggressive and conservative C2 styles. 


D. MEASURES OF EFFECTIVENESS 


The measures used to evaluate SSIM CODE’s decision-making capabilities are 
described in this section. The face validation is framed with the questions: 

" Does SSIM CODE arrive at realistic decisions? 

"Do the decisions support a commander’s intent? 


® [s tactical decision-making depicted realistically? 
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1. Does SSIM CODE Arrive At Realistic Decisions? 


Measuring the degree of realism in SSIM CODE is subjective. A face validation 
evaluates whether the tactical decisions generated by SSIM CODE are similar to those 
typically made by tactical commanders. Comparing SSIM CODE’s results to the 
analytical models supports the face validation. The analytical models presented in 
Chapter III highlighted the commander’s attributes and his estimate of the situation as the 
key elements in tactical decision-making. The face validation also examines how 
elements of the model influence quantitative measures of effectiveness (MOEs). 


2. Do the Decisions Support a Commander’s Intent? 


The battalion commander’s intent is represented as a goal (1:3 Blue-to-Red force 
ratio) with an associated time constraint (by the end of the CAS period). To determine 
whether the result of a SSIM CODE run meets the battalion commander’s intent, the 


force ratio and the time to complete the mission are used as MOEs. 


The Blue-to-Red force ratio at each TAI is measured at the end of each simulation 
run. A force ratio of 1:3 is specified by the battalion commander’s intent. This MOE 


allows for a quantitative analysis of the commander entity’s decision-making. 


An adjusted force ratio is used to capture a more robust range of success or 
failure. The adjusted force ratio penalizes the commander for leaving TAIs undefended. 
This measure may take on negative values. When Blue forces are engaging Red forces at 


number of blue forces 





a TAI, the adjusted force ratio is simply: adjusted force ratio = 
number of red forces 


(as before). However, when Red forces are present in a TAI that is undefended by Blue 
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force: adjusted force ratio = a . If Blue forces defend at a TAI that 
number of red forces 





is unoccupied by Red, the force ratio is set to zero. Finally, to measure the overall 
adjusted force ratio (across all TAIs) a battle force ratio (the response variable) is 


computed: 


3 
>) adjusted force ratio at TAI 





battle force ratio = ; - 
number of TAIs occupied by red forces 


The battle force ratio (BFR) is normalized by the number of TAIs occupied by the 
Red forces and accounts for the use of only one or two TAIs by Red forces. This 
response variable also penalizes the commander for defending TAIs that are not occupied 
by Red forces. The difference between measuring the commander’s success with a 


traditional force ratio and using a BFR is best illustrated by an example. 


If Red forces send one company (or three platoons) to each TAI and Blue sends 


all three platoons to BP 1, the forces are arranged as shown by Table 3: 








Blue Platoons a Gace ope 
BP 1 111 3 TAI 1 
BP2 - 3 TAI 2 
oe ; 3 TAI3 














Table 3. Sample Force Disposition 


Using traditional force ratio computation, the force ratios for TAIs 1, 2, and 3 are 
3 0 0 
3.3 +3 


The average force ratio is then = 0.333. Numerically, this 
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average force ratio meets the battalion commander’s requirement; yet, Red is able to 
occupy two TAIs unopposed. The traditional calculation does not penalize the Blue 


commander for this tactical error. 


The adjusted force ratios for TAIs 1, 2, and 3 are . =, and +. The BFR is 


a os | 
+ 


pea ee! 
3 3 _3 =0111. This force ratio is more representative of the situation as a measure 


of success (vice measuring force levels). 


Appendix C lists BFRs for the one hundred possible combinations of Blue and 
Red force dispositions. These combinations span the set of situations in which all three 
Blue platoons reach a BP and three Red companies are arranged in the TAIs. The 
dispositions possible when one or more Blue platoons do not arrive at a BP by the end of 


the simulation are not listed. However, the BFR measures these possibilities as well. 


When each of the three Blue platoons reaches a BP by the end of the CAS period, 
the range of the BFR is (-0.250, 0.333), as shown in Appendix C. However, when the 
possibility of a Blue platoon not reaching an assigned BP by the end of the CAS period is 
considered, the range of BFR is (—1.000, 0.333). The worst case is when one Red 
company is deployed to each of the three TAIs and no Blue platoons reach a BP. The 


ial ok 
adjusted force ratio in this case is —0.333 at each TAI. The BFR is = 2 =-1.000. 





BFR is used to measure the quality of decision-making in SSIM CODE 
simulation runs. BFR is the response variable in a factorial design experiment that 


examines the main effects and interactions of various SSIM CODE elements. 
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3. Is Tactical Decision-Making Depicted Realistically? 


To quantify the effect of time on tactical decision-making in SSIM CODE, time is 
first parameterized then evaluated as part of an MOE. Various time constraints (CAS 
periods) are examined. CAS periods of different time lengths are examined in various 
simulation experiments. For different time constraints, changes in the quality of 
decision-making are measured by evaluating the BFR attained at the end of each 
simulation run. The quality of the tactical decision-making is measured with BFR in 


each of these simulations. In these cases, time is a simulation parameter. 


F . ; BFR : 
Then, in a separate simulation, the MOE — - —— is 
Time Required to Complete Mission 





analyzed. The time constraint is removed and commander entities are allowed all the time 
required to deploy their forces in response to enemy actions. In this simulation, the MOE 
is used to measure the quality and efficiency of decision-making. A factorial design is 
used to evaluate the effects of the experiment factors on the MOEs. 


E. FACTORIAL DESIGN 


In a two-level factorial design experiment, a set of variables or factors is selected 
and two levels are fixed for each factor. The experiment runs include all possible 
combinations of the factor levels. Factorial designs are useful for evaluating the effect of 


each factor on a response variable. (Box, Hunter and Hunter, 1978) 


The SSIM CODE test scenario involves three decision factors (enemy, 
environment, own forces), three commander’s characteristics (C2 philosophy, C2 style, 


and experience) and three enemy AAs (use each of AA 1, 2, 3 or not). The response is 
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the BFR at the end of the CAS period (with time constraint) or BFR / Time to Complete 


Mission (no time constraint case). Table 4 lists the levels for each design factor. 



































Design Factor Level 
Label Description ae z 
A Environment Decision Factor Favorable | Unfavorable 
B Red (Enemy) Forces Decision Factor | Strong Weak 
c Blue (Own) Forces Decision Factor | Positive Negative 
D Cdr’s C2 Philosophy Mission Detailed 
E Cdr’s C2 Style Aggressive | Conservative 
F Cdr’s Experience Level High Low 
G Enemy AA 1 Use Not Use 
H Enemy AA 2 Use Not Use 
J Enemy AA 3 Use Not Use 




















Table 4. Levels of Each Design Factor in Experiment Design 


The response in each factorial design experiment corresponds to an MOE (either 
BFR or BFR/Time to Complete the Mission). The effect of each factor is the measured 


change in the response as the factor changes between its low and high levels. 


To examine whether SSIM CODE’s decision-making supports the commander’s 
intent, a quantitative evaluation of the relationship between design factors and the MOEs 


65 


is conducted in two phases. First, a main effects screening is completed using all the 
factors in Table 4. This analysis is focused on the main effect of each factor and does not 
consider interactions. Then, after determining the factors that have significant effects on 
BFR, a second factorial design is used to examine both main effects and first-order 


interactions on both MOEs. 


To complete the face validation, significant main effects and interactions are 
evaluated based on a reasonable approach to tactical decision-making. The relationships 
between factors and BFR are compared to what would be expected in a typical tactical 
situation. Model diagnostics are applied in each phase of the evaluation to examine the 
validity of assumptions made in the model settings. This procedure and its results are 
detailed in Chapter VI. 


1. Main Effects Screening Design 


Fractional factorial design experiments include only a fraction of the runs (factor 
level combinations) in a full factorial design. According to Box, Hunter and Hunter 
(1978), there are three main applications for fractional factorial designs: when screening 
a large number of factors for a subset of significant factors, in cases where certain 
interactions can be assumed negligible, and when groups of experiments are performed in 
sequence to resolve ambiguities. All three applications pertain to the main effects 


screening of SSIM CODE. 


The nine design factors in Table 4 are screened for significance. Two-factor 
interactions may provide useful insight. However, the confounding of two-factor 
interactions is acceptable in preliminary evaluations or screening experiments (Box, 


Hunter, and Hunter, 1978). Experiments to support the face validation are performed in 
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sequence. First, the significant main effects are identified, then a new factorial design is 


used to examine two-factor interactions. 


Three-factor interactions and above cannot be readily interpreted in terms of 
tactical decision-making. Furthermore, it is expected that these interactions will be 
negligible compared to main effects. According to Montgomery (1997), the sparsity of 
effects principle can be invoked to assume that main effects and low-order interactions 


dominate a system with many factors. 


Therefore, a resolution IV design is required for the initial screening of main 
effects. “A design of resolution...IV does not confound main effects and two-factor 
interactions [with each other], but does confound two-factor interactions with two-factor 


interactions...” (Box, Hunter and Hunter, 1978) 
A 2°“ fractional factorial design has resolution IV and is a | fraction of the full 


2° factorial. A full-factorial design would require 2° or 512 runs per replication to 
capture the effects of each factor and all possible interactions. The fractional factorial 


design requires thirty-two runs per replication. 


Appendix D summarizes the fractional factorial design by indicating each factor’s 
level for each of the required thirty-two runs per replication. Appendix D also lists the 
design layout, design generators and confounding patterns, as described by Box, Hunter 
and Hunter (1978). The design for the post-screening experiment is discussed in a later 


section. 
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Each replication consists of thirty-two runs. The factors are varied between runs 
as described by Appendix D. All other elements of the simulation remain constant from 
run to run. The tactical scenario, decision rules, commander’s decision probabilities and 
report probabilities are each identical from run to run. Uncertainty and randomness are 
introduced by the commander’s decision outcome choices, the detection error at the NAIs 


and TAIs, and the delays between OODA loop phases. 


Decision factors are represented by reports to the commander with an associated 
probability. The (+) factor level represents a 0.75 probability that the decision factor is in 
the state associated with the (+) level in Table 3. The (—) factor level represents a 0.25 
probability that the decision factor is in the (—) state. For example, the factor levels for 
the environment decision factor are defined as: 


+  Pf{Environ = favorable} = 0.75 <— P{Environ = unfavorable} = 0.25 
- P{Environ = unfavorable} = 0.75 < P{Environ = favorable} = 0.25 


2. First-Order Interaction Design 

Once significant main effects are determined, an appropriate factorial design is 
selected to estimate main effects and first-order interactions for the factors that were 
significant in the screening experiment. In this phase, an appropriate factorial design is 
selected to prevent confounding two-factor interactions with either main effects or other 
two-factor interactions. This requires either a resolution V fractional factorial design or a 


full factorial design. Chapter VI details the selection of this design. 
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VI. RESULTS AND ANALYSIS 


A. PILOT EXPERIMENT RESULTS 

An initial estimate of the BFR mean and variance is required to determine the 
number of replications necessary. These estimates were obtained through a pilot 
experiment that applies the 2°“ fractional factorial design. The Blue CAS period is set to 
one hour (simulation time). Thirty replications of the thirty-two runs (960 total runs) are 
conducted. Appendix E lists sample raw results for the first three replications (96 runs) 
of this pilot experiment (it is impractical to list all of the pilot results). The raw results 


include run number, factor levels, and BFR response for each run. 


The results for the pilot run are analyzed using S-plus (MathSoft Inc., 1999). 
Appendix F lists the S-plus code used for analysis. The pilot experiment estimated the 
mean BFR as -0.058 and the BFR standard error as 0.010. 

B. REPLICATIONS AND POWER CALCULATIONS 


The model setting for this experiment is: 


Vik =U Gt Ey 


where 

Vik = response observation for i" treatment, j" replication, kK" run 
LL = true mean 

q = treatment effect, i= 1,..., total treatments 

&j = errors, assumed to be i.i.d. Normal(0, o ) 


There are nine treatments in the first experiment. The treatments correspond to 
the factors listed in Table 4. This experiment tests a set of nine (one for each treatment) 
null hypotheses (H,). For each treatment, H, is: the treatment has no effect on the 
response (H,: T = 0). The alternate hypothesis (H,) for each treatment is that the 
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treatment has a significant effect on the response (H,: T # 0). A multi-factor analysis 
of variance (ANOVA) is conducted to test whether the treatment effects on the response 


are significant. 


The ANOVA considers the deviation from the mean response for each 
observation. The total sum of squared deviations (SST) and the sum of squared 
deviations associated with each treatment (SS7;,) are calculated. The sum of squared 
deviations due to error (SSE) is the difference between SST and all the SST, values. The 
mean squared for treatment (MST;,) is calculated by dividing SST, by the appropriate 
degrees of freedom (df). For treatment i, this calculation is: MST,, ; = SST, ; / df. The 
mean squared for error (MSE) is determined in a similar manner. The ANOVA uses the 


test statistic MST,/MSE. (Devore, 1995) 


If H, is true, the test statistic has an F distribution. An F-test is used in the 
ANOVA to determine whether the measured test statistic is likely to have come from an 
F distribution. When the test statistic has a value typically found in the F distribution, the 
ANOVA test procedure adjudicates H, as true. The probability that the test statistic is an 
observation from the F distribution is determined. If this probability is greater than a pre- 
set significance level, there is no detectible treatment effect. In this case, the variability 
associated with the treatment can be reasonably attributed to experimental error, and H, is 
adjudicated as true. If the F-test produces a probability that is less than the experiment’s 
significance level, then the effect of the treatment on the response cannot reasonably be 
attributed to error and Hy, is rejected. In this case, the treatment is deemed to have a 


significant effect on the response variable. (Devore, 1995) 
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The p-value (associated with a test statistic observation) is the probability of 
observing a value of the test statistic as extreme or more extreme than the observed one, 
assuming H, is true. The p-value is also the minimum significance level at which Ho is 


rejected, given realized value of the test statistic. (Devore, 1995) 


The objective of hypothesis testing is to determine whether deviations from the 
true response mean are due to chance variation or if these deviations are associated with a 
particular factor. Two types of errors may occur. Type I error occurs if Ho is rejected 
when it is true. Type II error occurs when Hy is accepted but false. The probability of 


type I error is set by the experiment’s significance level (a). (Devore, 1995) 


Type II error reflects the sensitivity of the analysis when H, is true. The 
probability of Type II error is denoted by B. The power of a hypothesis test is 1-B. When 
a is fixed, B can be improved by increasing the number of replications or sample 
observations. If the response variance can be estimated and a level for detectible 
deviations from the response mean is set, the number of replications required to attain a 


specific B can be determined by using power calculations. (Devore, 1995) 


Figure 17 shows power curves based on response variable mean, response 
variance, desired detectible deviation from the mean, and the number of treatments. The 
S-plus code used to generate the curves in Figure 17 is based the power calculation in the 
thesis Agent Based Simulation as an Exploratory Tool in the Study of the Human 


Dimension of Combat (Brown, 2000). This code is included in Appendix G. 


a 





The pilot run provides estimates of the BFR mean and_ variance: 


Ul = -0.058, o° = 0.010° = 0.0001. A significance level of 0.01 is used in this 
experiment. To detect a deviation (t) of five-percent or more from the estimated mean 
BFR, the level of detectible deviations from the mean is_ set at: 
r=lul * 0.05 = 0.058 * 0.05 = 0.0029 = 0.0025. From Figure 17, it takes sixty 


replications to detect such a deviation with power = 0.99. 








Replications Required 





0.0010 0.0015 0.0020 0.0025 


tau = Deviations from Mean Battle Force Ratio 








Figure 17. Power Curves for the 2°* Fractional Factorial Design 


c; PHASE 1: MAIN EFFECTS SCREENING 


Sixty replications of thirty-two runs (1920 total runs) were conducted to screen 
the main effects for significance. The raw results were analyzed using the multi-factor 
ANOVA and the S-Plus code in Appendix F. The ANOVA is summarized in Table 5. P- 


values are listed in the last column. 
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ANOVA 
(Response Variable: BFR) 








Factor DE Sum of Sq Mean Sq F Value Pr (F) 

E 1 Lew29O05 1,29053 14.1408 0.0001747 

R 1 2.0129 2.01286 22.0555 0.0000028 

B 1 3.4642 3.46422 37.9585 0.0000000 
C2P 1 54:8: 95A. 5.89510 64.5944 0.0000000 
C2s 1 2.1780 2, 17801 23.8651 0.0000011 
EXP i 30.3762 30.37617 332.8406 0.0000000 
AA aL 0.0026 0.00257 0.0282 0.8666986 
AA2 1 0.1380 0.13800 1.5121 0.2189642 
AA3 1 0.0011 0.00109 O2O0T1o. 0. 9131193 


Residuals 1910 174.3131 0.09126 





Table 5. Results of Main Effects Screening Experiment ANOVA 
CAS Period = 60 min. 


1. Main Effects Significance 


The ANOVA of the main effects screening experiment shows that all the three 
decision factors and the three commander attributes are highly significant (i.e., p-value 
<< 0.01) ata = 0.01. The effects of the AAs used by Red are not significant. It is also 
clear that the F-test statistics for the three AAs differ. This asymmetry in the ANOVA 
for the AAs is due to the fact that each AA is observed differently within the scenario. 
Use of AA3 can be determined by observing enemy movement to the south at NAII 
(Figure 16). However, use of AAI or AA2 can only be determined at NAI 2. Defense 
against the use of AA2 is the least difficult since its central location allows rapid 
allocation of forces from either the assembly area, BP1 or BP3. However, movement to 


defend against use of AA1 or 3 requires more time. 


According to Law and Kelton (2000), during factor screening, once a factor is 


deemed irrelevant, it should be fixed at a reasonable level and omitted from further 
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consideration. In subsequent experiments, the AAs are not included as factors; instead, 


they are fixed (all Red Companies move to the center TAI using AA2). 


The commander experience factor dominates the variance of BFR. The MSE for 
experience is an order of magnitude larger than for the other five main effects. Further 
analysis and interpretation of significant main effects are discussed in later sections. 


2. Fractional Factorial Model Diagnostics 


The adequacy of this model was analyzed to determine if the residuals have a 
mean of zero, if the normality assumption for model errors is valid, and if there are any 
points that exert high leverage. The model diagnostics are illustrated in the following 


figures. 


Figure 18 illustrates that the variance in the residuals does not differ greatly from 
factor to factor. The median residual is slightly greater than zero, which indicates a slight 
negative skew, assuming that the residual mean is zero in each case. There are many 
more negative outliers than positive outliers. This confirms the presence of a negative 


skew in the residuals. 
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Figure 18. Box Plot of Residuals for each Factor Level 
CAS Period = 60 min; Fractional Factorial Design. 


The quantile-quantile (Q-Q) plot, Figure 19, indicates that the residuals are close 
to normal, but the left tail is larger than normal and the right tail is smaller. This 


characteristic also indicates the presence of a negative skew in the residuals. 


ees. 








Residuals 





-2 0 2 
Quantiles of Standard Normal 








Figure 19. Q-Q-Plot of Residuals 
CAS Period = 60 min; Fractional Factorial Design. 


Figure 20 shows Cook’s distance for each observation. Cook’s distance measures 
the leverage and influence of a single observation on the whole model. A point is 
considered influential when it’s Cook’s Distance exceeds 1.0 (Hamilton, 1992). Cook’s 
distance is well below 1.0 for each observation. This indicates that there are no high 


influence data points and thus no high leverage points. 
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Figure 20. Cook’s Distance 
CAS Period = 60 min; Fractional Factorial Design. 


Figure 21 is a histogram of the residuals with a standard Normal curve 
superimposed. This graph shows that the residuals are near Normal, but clearly have a 


negative skew. 
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Figure 21. Histogram of Residuals 
CAS Period = 60 min; Fractional Factorial Design. 


a7 








Close examination of the histogram’s shape indicates that perhaps the residuals 
are bi-modal. There appears to be a small group of residuals, located in the left tail of the 
Normal curve, with a mean of —0.75. This would indicate that the BFR data examined 


come from two different distributions. 


Because the simulation was stopped after the one-hour CAS period, some 
commander entities had not completed their tactical decision-making and deployment of 
forces. Thus, this set of observations is censored data. A histogram of the BFR values 
(Figure 22) confirms this perception. The grouping of BFRs between 0.0 and 0.4 
represent commander entities that have completed their mission. The group with 
negative BFRs is likely to have forces still in motion. Thus, the Blue platoons have not 


reached their BPs and the adjusted force ratio values at the TAIs have negative values. 
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Figure 22. BFR Occurrences for 1,920 Simulation Runs 
CAS Period = 60 min; Fractional Factorial Design. 
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To compensate for the slower decision-making in the censored observations, the 
CAS period was parameterized over values greater than one hour in subsequent runs. 
Also, one experiment was conducted after removing the time constraint. This allowed all 


commander entities to complete their mission. 


In general, the model conforms to the ANOVA assumptions. No single point 
exerts excessive influence or leverage. The residuals have a mean near zero and do not 
deviate greatly from normality. The residuals are skewed to the left due to censored 
observations. The assumption of error terms distributed as independent, identically 
distributed (i.i.d.) Normal (0, 0°) appears to hold, but the residuals should be examined 
again with a longer CAS period. 

D. PHASE 2: FULL FACTORIAL DESIGN 

The next step is choosing a factorial design to examine main effects and first- 
order interactions among the six significant factors. Since it is not known which two- 
factor interactions are significant, all possible interactions between pairs of the six 
remaining factors must be considered. A full factorial design with six factors requires 2 
or sixty-four runs per replication. The model setting is: 


Vik = M+ OG + OF + Ej 


where 

Yijk = response observation for i” treatment, j" replication, k" run 
HL = true mean 

T = treatment effect, i= 1,..., total treatments 

0; = interaction variable, j = 1,..., total replications 

&jk = errors, assumed to be i.i.d. Normal(0, 0”) 


k = 1,..., total runs 
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This experiment tests a null hypothesis for each effect and for each interaction. 
Each individual null hypothesis states that the factor (environment decision factor, Red 
decision factor, Blue decision factor, commander’s C2 philosophy, commander’s C2 
style, or commander’s experience level) or first-order interaction has no significant effect 
on the response (BFR or BFR/Time to Complete Mission). ‘The alternative hypothesis is 
that the individual factor or interaction has a significant influence on the response. The 
design of this experiment allows analysis of two-factor interactions without confounding. 
Appendix H describes the layout of this design. There are six factors and fifteen (six- 


choose-two) interactions for a total of twenty-one hypothesis tests. 


A similar procedure is followed in the second phase of the SSIM CODE 
evaluation as in the main effects screening. First, a pilot run for a six-factor design is 
used to estimate the mean and variance of BFR. These estimates are used in a power 
calculation to determine the number of replications required. Appendix I includes the 
estimates from the pilot run and the power curve graphs. With six factors and sixty-four 
runs per replication, thirty replications (1,920 total runs) allow the detection of a three 


percent, or greater, deviation from the mean BFR. 


First, the time constraint is removed. Then a replicated simulation is conducted 
for each of five CAS periods (two through six hours). After each simulation, statistical 
analysis is accomplished using S-plus. An ANOVA is conducted, and the effects of the 


factors and first-order interactions are calculated. 
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1. BFR MOE in No Time Constraint Case 


Two MOEs~ were’ evaluated in this simulation: BFR and 


ERS . Table 6 summarizes the results for the BFR MOE. 





Time Required to Complete Mission 





























ANOVA 
(Response: BFR) 

Factor D£ Sum of Sq Mean Sq F Value Pr (F) 

E 1 0.125130 0.125130 41.345 0.0000000 

R 1 0.151506 0.151506 50.060 0.0000000 

B 1 0.209492 0.209492 69.219 0.0000000 

C2P 1 3.997764 3.997764 1320.915 0.0000000 

c2s 1 0.484505 0.484505 160.087 0.0000000 

EXP 1 2.034505 2.034505 672.228 0.0000000 

E:R 1 0.000040 0.000040 0.013 0.9082729 

E:B 1 0.007216 0.007216 2.384 0.1227264 

E:C2P 1 0.068880 0.068880 22.759 0.0000020 

E:C2S 1 0.009531 0.009531 3.149 0.0761269 

E:EXP 1 0.017054 0.017054 5.635 0.0177054 

R:B 1 0.010032 0.010032 3.315 0.0688134 

R:C2P 1 0.088775 0.088775 29.332 0.0000001 

R:C2S 1 0.000040 0.000040 0.013 0.9082729 

R:EXP 1 0.017054 0.017054 5.635 0.0177054 

B:C2P 1 0.145642 0.145642 48.122 0.0000000 

B:C2S 1 0.011074 0.011074 3.659 0.0559160 

B:EXP 1 0.039624 0.039624 13.092 0.0003043 

C2P:C2S 1 0.550130 0.550130 181.770 0.0000000 

C2P:EXP 1 1.782422 1.782422 588.936 0.0000000 

C2S:EXP 1 0.141797 0.141797 46.852 0.0000000 

Residuals 1898 5.744319 0.003027 
Interactions 
Effects se 
Main Effects E:R 0.00028935 0.002511 
E:B -0.00387731 0.002511 
Effects ae E:C2P -0.01197917 0.002511 
E 0.01614583 0.002511 E:C2S -0.00445602 0.002511 
R -0.01776620 0.002511 E:EXP -0.00596065 0.002511 
B 0.02089120 0.002511 R:B 0.00457176 0.002511 
C2P 0.09126157 0.002511 R:C2P 0.01359954 0.002511 
C28 -0.03177083 0.002511 R:C2S 0.00028935 0.002511 
FXP 0.06510417 0.002511 R:EXP 0.00596065 0.002511 
B:C2P -0.01741898 0.002511 
B:C2S -0.00480324 0.002511 
B:EXP -0.00908565 0.002511 
C2P:C2S 0.03385417 0.002511 
C2P:EXP -0.06093750 0.002511 
C2S:EXP 0.01718750 0.002511 
BFR 
Mean = 0.285 Standard Error = 0.001 





Table 6. BFR Results Without Time Constraint 
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All of the main effects are highly significant (i.e., p-value << 0.01) at the a = 
0.01 level. C2 philosophy dominates the MSE in the model. This is a different result 
than in the screening experiment. With a time constraint (in the screening), experience 


level is the dominant factor. Now, without a time constraint, C2 philosophy dominates. 


The MSE of each of the three commander attributes is at least one order of 
magnitude larger than the MSE for each decision factor. This indicates that commander 
attributes have more effect on BFR than decision factors. These observations are 


depicted in Figure 23, which shows the average effect of each factor on BFR. 
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Figure 23. Main Effects and Two-Factor Interaction Effects on BFR 
No Time Constraint. 
Main effects and interactions significant at @ = 0.01 are depicted. 


Each significant factor has a positive effect on BFR except for the Red decision 


factor and C2 style. Because these factors are involved in significant interactions, their 
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effects cannot be determined directly from their magnitude and sign—the interactions 


must be considered. 


C2 philosophy has a significant interaction with each of the other factors. 
Additionally, the C2  style-commander’s experience interaction and the Blue- 
commander’s experience interaction are significant. Significant main effects must be 
interpreted along with any associated significant interactions. According to Law and 


Kelton (2000), the combined effect of two interacting factors can be computed as: 


: é1 2 E12 
Combined Effect = 5 Xit+ 5 X2 + ; XiX2 





where 

e; = main effect of factor 1 

é€2 = main effect of factor 2 

€12 = interaction effect of factors 1 and 2 

x; =leveloffactor 1 (high= +1 and low =-1) 
X2 = level of factor 2 


The C2 philosophy-environment interaction is shown in Table 7. (This is a two- 
way factor table in the style of Box, Hunter and Hunter (1978).) Regardless of whether 
the environment is favorable or unfavorable, commanders with mission C2 philosophy 
have a higher BFR than those with detailed C2 philosophy. When the commander has a 
mission C2 philosophy, BFR is above the mean. For commanders with mission C2 
philosophy, BFR is affected about the same by a favorable or unfavorable environment 
decision factor. However, when C2 philosophy is detailed, an unfavorable environment 


reduces BER by twice the amount than a favorable environment does. 
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Mission (+) 








| 0.044 0.048 | 
C2 Philosophy _ | Mean BFR = 0.284 | 
| -0.060 -0.032 | 
Detailed (-) 
(-) ; (+) 
Unfavorable Prenton Favorable 


Table 7. Interaction Between C2 Philosophy and Environment 


Table 8 shows the C2 philosophy-Red interaction. Regardless of the Red (enemy 
forces) decision factor, commanders with mission C2 perform better than those with 
detailed C2. When the commander has a mission C2 philosophy, BFR is affected about 
the same by a weak or strong Red decision factor. However, when C2 philosophy is 


detailed, a strong enemy reduces BFR by twice as much. 


Mission (+) 











0.048 0.044 
C2 Philosophy Mean BFR = 0.284 
-0.030 -0.062 
Detailed (-) 
=) (+) 
Weak fed Strong 


Table 8. Interaction Between C2 Philosophy and Red 


Table 9 shows that a mission C2 philosophy results in a higher BFR, regardless of 
the Blue (own forces) state. Given a mission C2 philosophy, the effects on BFR of 
positive and negative Blue decision factors are nearly equal. When C2 philosophy is 


detailed, positive Blue forces result in a notably higher BFR than negative Blue forces. 


Mission (+) 








| 0.044 0.048 | 
C2 Philosophy _ | Mean BFR = 0.284 | 
| -0.064 -0.027 | 
Detailed (-) 
(-) (+) 
Negative pie Positive 


Table 9. Interaction Between C2 Philosophy and Blue 
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Table 10 describes the commander’s experience-Blue decision factor interaction. 


High experience results in a BFR above the mean, regardless of the Blue decision factor. 











High (+) 
0.027 0.039 
Experience Mean BFR = 0.284 
-0.048 -0.018 
Low (-) 
(-) (+) 
Negative mae Positive 


Table 10. Interaction Between Experience and Blue 


According to Table 11, given mission C2 philosophy, there is almost no 
difference between a conservative or aggressive C2 style. However, given detailed C2 
philosophy, a commander entity with aggressive C2 style performs much worse than one 


with conservative C2 style. 


Mission (+) 








| 0.045 0.047 —s«| 
C2Philosophy | Mean BFR = 0.284 | 
| -0.013 -0.079 | 
Detailed (-) 
i) (+) 
Conservative ee ovie Aggressive 


Table 11. Interaction Between C2 Philosophy and C2 Style 


Table 12 shows the C2 style-commander’s experience interaction. Given high 
experience, an aggressive C2 style results in a better BFR. If experience level is low, it is 


much better to have a conservative C2 style. 
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High (+) 








| 0.040 0.025 | 
Experience | Mean BFR = 0.284 | 
| -0.008 -0.057 | 
Low (-) 
(-) (+) 
Conservative be otyle Aggressive 


Table 12. Interaction Between C2 Style and Experience 


The interaction between-commander’s experience and C2 philosophy is described 


by Table 13. High experience results in a BFR above the mean, regardless of C2 


philosophy. Given a mission C2 philosophy, experience level makes a small difference 


in BFR. A combination of low experience level and detailed C2 philosophy has a very 


large negative effect on BFR. 











High (+) 
0.018 0.048 
Experience Mean BFR = 0.284 
-0.109 0.044 
Low (-) 
= . + 
Seni ee Pilloserny i 


Table 13. Interaction Between C2 Philosophy and Experience 


The mean BFR for the no time constraint case (0.284 with a standard error of 
0.001) is close to the goal of 1:3 or 0.333. This indicates that a majority of commander 
entities completed their mission successfully. A histogram of BFRs (Figure 24) shows 
that most commanders attained the 1:3 force ratio specified in the battalion commander’s 
intent. A few commander entities, however, still performed poorly even though there 
was no time constraint. This could have been caused by a misinterpreted force ratio goal 
(commander’s intent interpretation was a function of experience level) or by other 


combinations of factors that resulted in poor decision-making and a low BFR. 
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Figure 24. Occurrences of BFR Values for 1,920 Simulation Runs 
No Time Constraint. 


Even though some BFRs are low, there is no censored data. All the commander 
entities completed their decision-making and allocation of forces. The BFR values 
indicate distinct groupings of commander entities that attained the BFR goal and those 
that fell short. However, because these observations are not censored, the error structure 
in the model is closer to Normal (0, 0). Figure 25 shows that the residuals for this case 


are closer to Normal than for the screening experiment. 


87 











Frequency 











Residuals 








Figure 25. Histogram of Residuals Compared to Standard Normal Curve 
No Time Constraint. 


2. BFR/Time MOE in No Time Constraint Case 


Table 14 summarizes the results of the no time constraint case in terms of the 


BFR 
Time Required to Complete Mission 


MOE. This MOE is described in force ratio per 





hour units. 


88 





ANOVA 


(Response: BFR / Time to Complete Mission) 





























Factor D£ Sum of Sq Mean Sq F Value Pr (F) 

E 1 0.0016604 0.0016604 4.1390 0.0420435 

R 1 0.0081968 0.0081968 20.4326 0.0000066 

B 1 0.0062089 0.0062089 15.4775 0.0000865 

C2P 1 0.2998683 0.2998683 747.5020 0.0000000 

c2s 1 0.0047389 0.0047389 11.8129 0.0006009 

EXP 1 0.1721357 0.1721357 429.0943 0.0000000 

E:R 1 0.0003701 0.0003701 0.9225 0.3369402 

E:B 1 0.0000394 0.0000394 0.0982 0.7539758 

E:C2P 1 0.0021861 0.0021861 5.4495 0.0196773 

E:C2S 1 0.0000000 0.0000000 0.0000 0.9990761 

E:EXP 1 0.0012140 0.0012140 3.0262 0.0820926 

R:B 1 0.0004247 0.0004247 1.0588 0.3036204 

R:C2P 1 0.0059032 0.0059032 14.7153 0.0001291 

R:C2S 1 0.0006165 0.0006165 1.5368 0.2152433 

R:EXP 1 0.0008980 0.0008980 2.2386 0.1347672 

B:C2P 1 0.0066091 0.0066091 16.4749 0.0000513 

B:C2S 1 0.0002621 0.0002621 0.6533 0.4190194 

B:EXP 1 0.0008499 0.0008499 2.1186 0.1456884 

C2P:C2S 1 0.0088133 0.0088133 21.9695 0.0000030 

C2P:EXP 1 0.1027208 0.1027208 256.0591 0.0000000 

C2S:EXP 1 0.0092836 0.0092836 23.1419 0.0000016 

Residuals 1898 0.7614027 0.0004012 
Interactions 
Effects se 
E:R -8.7806e-004 0.00091419 
Main Fffects E:B -2.8655e-004 0.00091419 
E:C2P -2.1341e-003 0.00091419 
Effects se E:C2S -1.0588e-006 0.00091419 
E 1.8599e-003 0.00091419 E:EXP -1.5903e-003 0.00091419 
R -4.1324e-003 0.00091419 R:B 9.4069e-004 0.00091419 
B 3.5966e-003 0.00091419 R:C2P 3.5069e-003 0.00091419 
C2P 2.4995e-002 0.00091419 R:C2S 1.1333e-003 0.00091419 
C2S -3.1421e-003 0.00091419 R:EXP 1.3678e-003 0.00091419 
EXP 1.8937e-002 0.00091419 B:C2P -3.7106e-003 0.00091419 
B:C2S -7.3894e-004 0.00091419 
B:EXP -1.3306e-003 0.00091419 
C2P:C2S 4.2850e-003 0.00091419 
C2P:EXP -1.4629e-002 0.00091419 
C2S:EXP 4.3978e-003 0.00091419 
BFR / Time to Complete Mission 
Mean = 0.054 Standard Error = 0.0005 





Table 14. BFR/Time to Complete Mission Results 
No Time Constraint. 
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Except for the environment decision factor, the main effects are highly significant 
(p-value << 0.01) at the a = 0.01 level. For this MOE, C2 philosophy and commander’s 
experience dominate the MSE. The other factors have comparable MSE to each other but 


are an order of magnitude smaller than C2 philosophy and experience. 


C2 philosophy has significant interactions with each of the other factors, except 
with the environment. The C2 style-commander’s experience interaction is also 


significant in this case. 


The interpretation of the significant effects and interactions on this MOE is 
similar to the interpretation for the BFR MOE. Since the environment decision factor 
and its interaction with C2 philosophy are no longer significant at the a = 0.01 level, the 
implication is that commander entities are unaffected by the environment when attaining 
BFR/Time to Complete Mission. At a= 0.01, the environment significantly affects BFR, 
but not BFR/Time to Complete Mission. Thus, the effect of the environment is associated 


with the time to complete the mission. 


Figure 26 shows the average effect of each significant factor and interaction on 
BFR. The environment decision factor and its interaction with C2 philosophy are 
included for comparison with Figure 23. A similar relationship, between the effects of 
these factors and interactions, exists for the BFR MOE (Figure 23) and the BFR/Time to 


Complete Mission MOE (Figure 26). 
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Main Effect or Interaction 
Figure 26. Main Effects and Two-Factor Effects on BFR/Time 
No Time Constraint. 
The same main effects and interactions as in Figure 23 are shown for comparison. All of 
these effects, except E and E:C2P, are significant at a = 0.01. 





The time to finish the mission for each of the 1,920 runs is depicted in Figure 27. 
This statistic appears to be distributed exponentially. However, the set of observations do 


not pass a Chi-Squared goodness-of-fit test for the exponential distribution. 
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Figure 27. Histogram of Time to Complete Mission 
No Time Constraint. 
Mean = 6.9 hrs; Standard Deviation = 4.9 hrs. 
BFR : 
The MOE appears to be bi-modal. A group 





Time Required to Complete Mission 
of observations depicts efficient commander entities with an _ observed 


BFR 
Time Required to Complete Mission 


of 0.06 to 0.08. A second group has observed 





values between 0.00 and 0.04. The mean (0.054) is clearly not indicative of this MOE. 
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Figure 28. Histogram of BFR / Time to Complete Mission 
No Time Constraint. 

Mean = 0.054 force ratio/hr; Standard Deviation = 0.0005 force ratio/hrs. 

3. Effects of Varying CAS Period 

Next, the CAS period is parameterized in one-hour (simulation time) increments. 
From the previous experiment, 88% of the observed mission completion times are less 
than 11.0 hours. Red companies arrive at the TAIs between three and five hours after the 
start of the mission (they are spaced one hour apart). A six-hour CAS period starts when 


the third Red company arrives at a TAI and continues until 11.0 hours into the mission. 


With a six-hour CAS period, almost 90% of runs are completed missions. 


The CAS period was varied over five experiments from two to six-hours. This 
parameterization allows the analysis of factor effects over various time constraints. In 


this setting, the Blue commander must attain the desired BFR by the end of the CAS 
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period. Increasing the CAS period allows more time for the commander entity to observe 
the battlespace, make decisions, and act on those decisions. The mean BFR, main 
effects, and first-order interactions were examined for each CAS period setting. 
Appendix J summarizes the results for the various CAS periods. 


a. Mean BFR Over Five CAS Period Durations 


Figure 29 shows the relationship between mean BFR and CAS period. 
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Figure 29. Mean BFR vs. CAS Period 
2° Fractional Factorial Design. 
As commander entities are given more decision-making time, their performance 
(measured with BFR) improves. 


























The mean BFR clearly increases as the commander entity is allowed more 
time for information gathering, decision-making and implementing the decisions. As the 
CAS periods increase, the commanders with attributes that hinder success overcome their 
shortfalls and perform closer to the level of the better commanders. This effect makes 


tactical sense and is routinely demonstrated in practice. 
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b. Main Effects Over Different CAS Periods 
For each experiment, the main effects are significant. Figure 30 shows 


how the main effects vary with the CAS Period. 
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Figure 30. Main Effects on BFR for Various CAS Periods 


Compared to the effects of the commander attributes, the effects of the 
decision factors (E, R, and B) remain fairly constant across the range of CAS periods 
examined. The environment and Red decision factor effects decrease with longer CAS 
periods. However, the Blue main effect generally increases with CAS period. These 
trends represent a lesser impact on BFR of the enemy forces state and the environment 
when commander entities are given more time to decide and act. The state of own forces 


has more effect on the BFR as the mission times increase. 
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As CAS period increases, the effects of the three commander attributes 
clearly decrease. This indicates that—given enough time—the less experienced 
commanders (and those with less effective combinations of C2 philosophy and C2 style) 
perform closer to the level of commanders with a better set of attributes. These results 
make tactical sense. 


c. Significant Interaction Effects Over Different CAS Periods 


The interactions that were significant in the unconstrained experiment are 
also consistently significant throughout this set of five experiments. Figure 31 depicts 
significant interaction effects over the various CAS periods. All interaction effects, 
except the Blue-experience interaction, decrease with longer CAS periods. The Blue- 
experience interaction increases with CAS period because the Blue main effect is 
increasing with CAS period. The interactions are interpreted in a similar manner as in the 


unconstrained experiment. 
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Figure 31. Significant Interaction Effects on BFR for Various CAS Periods 


Figure 32 summarizes the effects of the six factors and the five significant 
interactions on BFR by showing the average magnitudes of factor effects. Figure shows 
that C2 philosophy, experience, and their interaction dominated the effect on BFR. Once 


again, the commander attributes were more influential than the decision factors. 


97 








0.300 





0.250 





0.200 





0.150 





Effect on BFR 


0.100 


0.050 








0.000 + 





E R B:EXP B:C2P C2S:EXP B C2S C2P:C2S C2P:EXP EXP c2P 


Factor 








Figure 32. Average Magnitudes of Effects on BFR 
Effect magnitudes are averaged over the five cases with different CAS periods. 
Standard error = 0.013. 
4. Full Factorial Model Diagnostics 
A diagnostic check of the model quality is conducted in the same manner as for 
the main effects screening. Each of the five experiments with differing CAS periods had 


similar model diagnostics. Figures 33 through 36 depict diagnostics for the full factorial 


model in the four-hour CAS period experiment (the central case). 


The full factorial model has better diagnostic results than the fractional factorial 
design. The box plots show fairly constant variance among the residuals at each factor 
level. The median for each box plot is near zero. Because the box plots are quite 


symmetrical, the mean will also be near zero. 
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Figure 33. Box Plot of Residuals for each Factor Level 
CAS Period = 240 min; Full Factorial Design. 
The residuals closely conform to the Normal distribution, according to the Q-Q 
plot. 
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Figure 34. Q-Q-Plot of Residuals 
CAS Period = 240 min; Full Factorial Design. 
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Figure 35 also reflects the near-normal residuals with a mean of zero. 
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Figure 35. Histogram of Residuals & Standard Normal Curve 
CAS Period = 240 min; Full Factorial Design. 


Cook’s distance is much smaller than 1.0 for each observation. Thus, there are no 


high influence or high leverage points. 
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Figure 36. Cook’s Distance 
CAS Period = 240 min; Full Factorial Design. 
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E. FACE VALIDATION 


1. Realism of Decision-Making 

The evaluation of SSIM CODE found tactical realism in the experimental results. 
Commanders improved their performance with time. Commanders with mission C2 
philosophy performed better in a scenario that required rapid allocation of forces in the 
face of uncertainty. Commanders with a detailed C2 philosophy performed better when 
given more time. Commanders with a high experience level performed much better than 
those with low experience levels, but given more time to decide and act, those with low 
experience performed closer to the level of those with high experience. The performance 


of the commander entity was not significantly affected by changing the enemy COA. 


The effects of the Blue, Red and environment decision factors changed less than 
those of the commander attributes, as the CAS period time was increased. However, the 
Blue decision factor increased with the CAS period while the other two decision factors 
decreased with the CAS period. The significant interactions have interpretations that 


make tactical sense. 


Time played an important role in the SSIM CODE evaluation. As commander 
entities had more time for decision-making, their performance improved. In the 
unconstrained experiment, most of the commander entities were able to attain the 1:3 
force ratio goal. In the five experiments with varying CAS periods, the BFR MOE 
clearly increased when commander entities had more decision-making time. 

2. Comparison to Analytical Model 

In Chapter III, the conditional probability model indicated that the commander 


attributes and the actual situation (decision factor states) would have the most influence 
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on the results of a tactical commander’s decisions. The six significant factors in the 
evaluation of SSIM CODE are the three commander attributes and the three decision 


factor states. These results agree with the analytical model. 


The detailed conditional probability model identified the commander attributes as 
the most influential factors in tactical decision-making. The results of the SSIM CODE 


evaluation also identified commander attributes as having the most effect on BFR. 
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VII. CONCLUSIONS 


A. THESIS OBJECTIVES 
As listed in Chapter II, the thesis objectives are: 


" Model tactical commander decision cycles (battalion and below). 
® Apply C2 doctrine. 
® Develop a functionality module for Combat XXI. 


® Exercise the SSIM CODE as a stand-alone simulation. 
® Evaluate the effectiveness of SSIM CODE’s decision-making. 


Each of these objectives was addressed in the development and evaluation of 


SSIM CODE. 


B. TACTICAL DECISION CYCLE MODEL 

SSIM CODE implements the OODA loop concept and includes inquiry-based, 
triggered, and directed information-processing in a representation of a_ tactical 
commander’s decision cycle. The commander traits applied by SSIM CODE are typical 
of those used to measure tactical commanders and their individuality: experience level, 
C2 style, and C2 philosophy. The decisions that SSIM CODE commander entities make 
(search, move and engage) develop tactical level courses of action. The results of the 
SSIM CODE evaluation demonstrate tactical sense and reflecte the nature of the 
analytical models. While the tactical decision cycle model in SSIM CODE can be 
developed further to improve its resolution and flexibility, this model is an effective first 


step in representing the decision cycle of a tactical commander. 


C. MARINE CORPS C2 PHILOSOPHY APPLICATION 
Marine Corps C2 philosophy was applied in SSIM CODE by distinguishing 


between mission and detailed C2. The results showed that mission C2 was more 
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successful than detailed C2, in the test scenario. In the cases with a shorter CAS period, 
commander entities with mission C2 had less accurate information, but were able to 
attain a higher BFR by accepting the limited information and making quicker decisions. 
Commander entities with detailed C2 used up time in improving their information and 
attained lower BFR values. However, when the CAS period was increased, commander 
entities had more time to decide and C2 philosophy had less of an effect. This is in 
keeping with the Marine Corps philosophy that mission C2 is effective when quick 


victories are required. 


D. SSIM CODE AS A STAND-ALONE SIMULATION 


SSIM CODE was employed effectively as a stand-alone simulation. For the 
evaluations, SSIM CODE was linked to Combat XXI through the SA module. The 
Combat XXI SA module served in dynamically updating the facts available to the 
commander entities and in allowing the commander entities a means to invoke the actions 
that resulted from decision-making. The next step in developing SSIM CODE as a part 
of Combat XXI C2 behaviors is to tightly couple SSIM CODE with other functionality 
modules (search, movement, communication, etc.) in Combat XXI. SSIM CODE can 
then be structured to meet all the abstract level requirements of Combat XXI 


functionality modules. 


E. EFFECTIVENESS OF SSIM CODE DECISION-MAKING 


Two stages of fractional factorial design experiments were used successfully in 
evaluating SSIM CODE. First, only main effects were evaluated to conclude that the 


enemy COA (combination of AAs) did not have a significant effect on BFR. However, 
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the other six factors had significant effects on BFR across the range of CAS periods 


considered. 


Next, two-factor interactions were examined with a full factorial design. All 
possible two-factor interactions of the six significant factors (decision factors and 
commander attributes) were considered. The main effects and interactions had 


reasonable interpretations. 


SSIM CODE was deemed to make tactical sense through a face validation. Its 
results reflected the analytical models described in Chapter III. The evaluation concluded 
that the first steps in developing a decision-making model for Combat XXI and the 
purpose of this thesis have been accomplished. Additional development will complete 
the task. 


F. ADDITIONAL DEVELOPMENT 
1. Extended Features and Applications 


Chapter I describes the process for developing a decision-making model for 


Combat XXI as: 


® Develop the concept of tactical decision-making for C2 into an analytical 
model. 


" Implement the decision-making model in a simulation coupled to Combat 
XXI’s behaviors package (loosely coupled with Combat XX1). 


® Evaluate the performance of the decision-making simulation compared to 
the analytical model. 


® Link the simulation to all applicable Combat XXI packages (tightly coupled 
with Combat XX1). 


® Enhance the abstract features of the simulation to handle all likely 
applications of Combat XXI1. 


The SSIM CODE model developed and evaluated in this thesis is merely an initial 


step that addresses the first three elements of completing a robust decision-making model 
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that is fully integrated with an operational version of Combat XXI. As Combat XXI is 


completed, SSIM CODE should expand in its capabilities. 


Expansion of SSIM CODE should address the final two steps in developing a 
decision-making model for Combat XXI. Full interaction with other Combat XXI 
modules (such as the search, movement, and engagement modules) should be completed. 
SSIM CODE’s abstraction level should equal the abstract capabilities of other Combat 


XXI components. 


Additional features for SSIM CODE should be considered. These features 
include expanding the number and levels of decision factors, implementing decision- 
making to support doctrinal targeting procedures, enabling the commander entity to 


develop and issue a commander’s intent. 


Another possible enhancement for SSIM CODE could be the dynamic updating of 
conditional probabilities associated with each commander entity. CCIRs could trigger 
decision cycles and also update the conditional probability distribution based on key state 
variables. For example, detecting an enemy force greater than a threshold size would not 
only initiate an engage decision, but would also change the probability that a commander 


entity would engage, given the decision factor stochastic state. 


Potential studies using SSIM CODE include: measuring the effects of degraded 
communication between the commander entity and other simulation entities, measuring 
the value of information accuracy by varying uncertainty in reports, and measuring the 
influence of a commander’s individuality by varying the commander entity’s attributes. 


Commander entities on all tactical levels within each participating force can employ a 
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SSIM CODE -based decision-making model. The interactions of multiple OODA loops 


can then be evaluated. 


SSIM CODE’s development can further enable the representation of non- 
linearity, intangibles, and co-evolving landscapes. The non-linear effects of tactical 
decisions by individual commanders on the outcomes of battles can be evaluated. 
Commander attributes can be expanded to include other intangibles such as fatigue and 
morale. Multiple instances of commander entities employing OODA loops can be used 
to analyze co-evolving landscapes. 


2. Model Validation 


In addition to the face validation in this thesis, a procedural validation could test 
the effectiveness of SSIM CODE. Such a procedural approach could consist of peer 
validation. A test scenario can be formulated as a tactical decision game and solutions 
from a sample of officers could be compared to SSIM CODE’s results. A complete 
validation would apply technical methods described in Validation and Verification of 
Knowledge-Based Systems (Gupta, 1991) and Verification, Validation and Accreditation 


of Army Models and Simulations (U.S. Army, 1999). 


A rigorous validation would include examining the data used to populate 
commander attributes and decision outcome probabilities. An assessment of the degree 
of realism associated with the data would be required. Conditional probabilities 
associated with commander entities could be developed in a broad sense. For example, 
all commanders of a certain service branch for a specific combatant would include 
similar data. These probabilities can be assigned with expert judgment based on doctrine, 


observed tactics and intelligence reports. 
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A more detailed estimation of commander decision probabilities and commander 
entity attributes could be derived from operational data and population surveys. For 
example, observations of tactical decision-making during exercises could be used to 


estimate probability distributions. 


Surveying tactical decision-makers (such as groups of company and field grade 
officers at resident professional schools) could lend insight into details that may be 
difficult to observe directly. Information relating to the hierarchy of decision factor 
importance for various situations could be obtained through surveying. Relationships 


between commander attributes and decision-making can also be examined in this manner. 


Experiments to directly attain values for decision-making probabilities or 
commander attributes would be an effective (albeit costly) means to estimate the data 
required by SSIM CODE. Small-scale experimentation could be applied to improve data 
sets used in SSIM CODE. Scenarios based on actual data with verified results either 
from experimentation or from validated simulations could be approximated using Combat 
XXI and SSIM CODE. The results could be used to determine the quality of the 
decision-making probabilities and commander attributes. Improvements to the data could 


be attained from an iterative approach. 


Perhaps the best sources for data to use in the SSIM CODE model are the various 
research efforts already in place in the Department of Defense modeling and simulation 
community. This work was detailed during a Military Operations Research Society 
workshop titled: Evolving the Practice of Military Operations Analysis in the 


Department of Defense. The Industrial College of the Armed Forces, the National 
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Ground Intelligence Center, and the Simulation Interoperability Standards Organization 
all have ongoing research efforts to provide data on tactical decision-making. The inputs, 
outputs, and processes involved in decision-making on the individual combatant level are 
being investigated (Burnett, Tamucci and Timian, 2000). Rule sets and characteristics to 
define tactical decision-makers are also being produced (Bjorkman, 2000). This data, 
developed specifically for use in combat simulations, can be used in the many potential 
applications for SSIM CODE and Combat XXI in the RDA and ACR domains. 


G. SSIM CODE APPLICATION 


SSIM CODE was developed as a Combat XXI functionality module and has a 
direct application in Combat XXI. Because Combat XXI is required to meet the 
Department of Defense high-level architecture (HLA) standards, it will be capable of 
exchanging inputs, outputs and models with other HLA compliant simulations. HLA 
enables simulation interoperability and reuse. This implies the potential application of 


the SSIM CODE model with other HLA-compliant simulations. 


Simulation results attainted with SSIM CODE can be applied directly to lower 
resolution simulations (regardless of HLA compliance). For example, one result from the 
SSIM CODE evaluation was that the mission completion time MOE could potentially be 
modeled with an exponential distribution. A more aggregated simulation may model 
several sequential missions, assigned to one commander, without adjudicating each 
battle. This type of simulation could apply the SSIM CODE result by using a Poisson 
process to model the number of completed missions. If the missions are completed 


independently, a counting process for completed missions with exponential mission 
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completion times (interarrival times) meets the requirements for a Poisson process (Ross, 


1997). 


Stochastic decision-making in Combat XXI has applications beyond the U.S. 
Department of Defense. The Australian armed forces currently use CASTFOREM for 
high-resolution combat simulation (Australian Defence Simulation Office, October 
2000). Combat XXI will be replacing CASTFOREM as Australia’s high-resolution 
combat simulation of choice. Improved C2 processes from SSIM CODE can serve to 


enhance Combat XXI applications in Australian modeling and simulation domains. 


The development of SSIM CODE resulted in over nine thousand lines of Java 
code. It is impractical to include the lengthy source code in this thesis. Also, source code 
related to Combat XXI is copyrighted. Therefore, the Java source for SSIM CODE is 
available by contacting Major S. Posadas on the Combat XXI development team, 
TRADOC Analysis Center - White Sands Missile Range, NM 88002. Appendix K 
provides an overview of the central classes in SSIM CODE. Methods and attributes for 


the classes are listed in the appendix. 
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APPENDIX A. TEST SCENARIO PARAMETERS 





Environment Parameters 



































Distance from Red Force Origin To NAT 1 10 km 
Distance from NAI 1 to NAI 2 20 km 
Distance from NAT 1 to TAI 3 40 km 
Distance from NAI 2 to TAI 2 20 km 
Distance from NAI 2 to TAI 1 20 km 
Distance from Blue Force Assembly Area To BP 1 35 km 
Distance from Blue Force Assembly Area To BP 2 25 km 
Distance from Blue Force Assembly Area To BP 3 35 km 











Red (Enemy) Force Parameters 

















Number of Companies 3 
Combat Forces per Company 120 
Average Speed 12 km/hr 
Separation in Time between Companies 1 hr 
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Blue (Own) Forces Parameters 











Number of Platoons 3 
Combat Forces per Platoon 40 
Average Speed 15 km/hr 





Standard Deviation for Detection Error at NAI 1 


0.15 * Number of Actual Forces 





Standard Deviation for Detection Error at NAI 2 


0.10 * Number of Actual Forces 





Standard Deviation for Detection Error at all TAIs 


0.05 * Number of Actual Forces 





Distance from Blue Force Assembly Area To BP 2 


25 km 








Distance from Blue Force Assembly Area To BP 3 





35 km 








Blue Company Commander Entity OODA Loop Phase Times 




















OODA Loop Phase Low Experience High Experience 
Observe* 10 min 3min 
Orient 10 min 3 min 
Decide 10 min 3 min 
Act 10 min 3 min 
Between OODA Loops 20 min 10 min 











*Commander Entities with Detailed C2 Philosophy Take Twice as Long While Requesting 


IMore Accurate Information 








Blue Company Commander Entity Deviation from Commanders Intent 





Low Experience 


High Experience 





Standard Deviation from 
Specified Force Ratio Goal* 








0.050 * Force Ratio Goal 0.025 * Force Ratio Goal 
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Blue Company Commander Entity Decision Probabilities 


Conditional Probability of Taking Primitive Action (Search, Move or Engage) Given Combined State 































































































Aggressive Conservative 
Combined State C2 Style C2 Style 
Search Decision 
1: FWP 0.999 0.750 
2: FSP 0.990 0.740 
3: FWN 0.980 0.720 
4: FSN 0.950 0.700 
5: UWP 0.920 0.670 
6: USP 0.850 0.600 
7: UWN 0.700 0.450 
8: USN 0.500 0.250 
Movement Decision 
1: FWP 0.999 0.750 
2: FSP 0.990 0.730 
3: UWN 0.980 0.720 
4: USN 0.950 0.700 
5: FWN 0.900 0.650 
6: FSN 0.800 0.550 
7: UWN 0.650 0.400 
8: USN 0.450 0.200 
Engagement Decision 
1: FWP 0.999 0.700 
2: UWP 0.980 0.470 
3: FSP 0.950 0.350 
4: USP 0.900 0.230 
5: FWN 0.800 0.180 
6: UWN 0.700 0.150 
7: FSN 0.550 0.120 
8: USN 0.300 0.100 
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Decision Probabilities 


The order of the combined states is from best to worst (left to right) according to 
the type of decision. This ordering is based on expert judgment. For example, FSP 
(favorable environment, strong enemy, and positive own forces) is a better state than 
FWN (favorable, weak, negative) when considering a search decision. This ordering 


reflects the importance of the environment, own forces and enemy forces, from highest to 
































lowest. 
Engage Decision 
A 
& 
ca) 
9) 
© 
=) 
= 
Ww ; 
5 A Aggressive 
2 fe) Conservative | 
i 
© 
2 
° 
hen 
ou 








1: FWP 2:UWP 3:FWN 4:UWN 5:FSP 6:USP 7:FSN_ 8:USN 
Combined State 


Engage Decision Probabilities 
Aggressive and conservative commanders have differing trends. The probability of 
engaging declines more rapidly for conservative commanders. Conservative 
commanders are 20 to 65% less likely to engage given the same combined state. 
Combined states are ordered to rank enemy forces, own forces, then the environment 
decision factors from most to least critical. 
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Search Decision 


























Probability of Search 





4 Aggressive 
° Conservative 

















1: FWP 2:FSP 3: FWN 4:FSN 5: UWP 6: USP 7: UWN 8: USN 
Combined State 


Search Decision Probabilities 
Aggressive and conservative commanders have the same trend. Conservative 
commanders are 20 to 25% less likely to search given the same combined state. 
Combined states are ordered to rank the environment, own forces, then enemy forces 
decision factors from most to least critical. 


Move Decision 





























Probability of Move 


A Aggressive 
° Conservative 

















1:FWP 2:FSP 3:UWP 4:USP 5: FWN 6:FSN 7: UWN 8: USN 
Combined State 


Move Decision Probabilities 
Aggressive and conservative commanders have the same trend. Conservative 
commanders are 25% less likely to move given the same combined state. Combined 
states are ordered to rank own forces, the environment, then enemy forces decision 
factors from most to least critical. 


121 


THIS PAGE INTENTIONALLY LEFT BLANK 


122 


APPENDIX B. TEST SCENARIO DECISION RULES 


Search Decision Rules 


1. If the search decision outcome was NOT to search, continue collecting information. 

2. Otherwise, 
a. Update the number of enemy detected at each NAI. 
b. If at least two-thirds of the total estimated enemy force has passed through NAT 1, 
search in NAT 2. 


Engagement Decision Rules 


1. If the engage decision outcome was NOT to search, continue collecting information. 
2. Otherwise, 
a. Update the number of enemy detected at each TAI. 
b. Update the number of enemy forces anticipated at each TAI. 
1) Update the TAIs randomly (each equally likely) 
2) Ifthe force ratio goal has not been met at a TAI, 
a) Task any available Blue forces already at the corresponding BP to engage. 
b) Task any available Blue forces in the assembly area to move to the proper 
BP and engage. 
c) Task any available Blue forces from another BP that has already met it 
force ratio goal to move to the appropriate BP and engage. 


d) If C2 style is aggressive, task up to three platoons to engage. 
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Movement Decision Rules 


1. If the move decision outcome was NOT to search, continue collecting information. 


2. Otherwise, 


a. Update the number of enemy detected at each NAI. 


b. Update the number of enemy forces anticipated at each TAI 


1) 
2) 


3) 


4) 


5) 


6) 


7) 


8) 


9) 


10) 


11) 


Update the TAIs randomly (each equally likely) 

If more than two companies have been detected moving north at NAI 2, expect 
3 enemy companies at TAI 1. 

If more than one company have been detected moving north at NAI 2, expect 2 
enemy companies at TAT 1. 

If at least one company has been detected moving north at NAI 2, expect 1 
enemy company at TAI 1. 

If more than two companies have been detected moving south at NAI 2, expect 
3 enemy companies at TAI 2. 

If more than one company have been detected moving south at NAI 2, expect 2 
enemy companies at TAT 2. 

If at least one company has been detected moving south at NAI 2, expect 1 
enemy company at TAI 2 

If more than two companies have been detected moving south at NAI 1, expect 
3 enemy companies at TAI 3. 

If more than one company have been detected moving south at NAI 1, expect 2 
enemy companies at TAT 3. 

If at least one company has been detected moving south at NAI 1, expect 1 
enemy company at TAI 3 

If Blue forces are not in place or enroute to the proper BP to achieve the 
desired force ratio, move one available platoon from the assembly area to each 


BP requiring additional forces. 
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APPENDIX C. BFR FOR 100 FORCE DISPOSITIONS 


Force dispositions are shown in terms of platoons. One Red company is 
represented by three Red platoons. The forces are arranged, from top to bottom, in BP 
1,2, and 3 (for Blue) or TAI 1,2,and 3 (for Red). BFR is shown below each 


corresponding force disposition. 































































































[piue Red |B1ue Red |B1ue Red |B1ue Red |B1ue Red |B1ue Red |B1ue Red |B1ue Red |B1ue Red |B1ue Red | 
toast | asa te ee a Se ee a i Se a a eee eS ae 
ra eae foe 00 ea (ie (eed fs eo We ee (ee (ee eee (ee ea eye Ue 
Dee hed et ee Ee ees I Sain Peep ae Un ae ae Dae ae om ad Fee | 
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APPENDIX D. 2°*RESOLUTION IV DESIGN 
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2 °“ Fractional Factorial Design Generators: 
F =BCDE G=ACDE H=ABDE J=ABCE 











































































































Factor or Confounded With 
Interaction 

I ABFG + ACFH + ADFJ + BCGH + BDGJ + CDHJ 

A BFG + CFH + DFJ + BCEJ + BDEH + CDEG + EGHJ 

B AFG + CGH + DGJ + ACEJ + ADEH + CDEF + EFHJ 

C AFH + BGH + DHJ + ABEJ + ADEG + BDEF + EFGJ 

D AF] + BGJ + CHJ + ABEH + ACEG + BCEF + EFGH 

E ABCJ + ABDH + ACDG + AGHJ + BCDF + BFHJ + CFGJ + DFGH 

F ABG + ACH + ADJ + BCDE + BEHJ + CEGJ + DEGH 

G ABF + BCH + BDJ + ACDE + AEHJ + CEFJ + DEFH 

H ACF + BCG + CDJ + ABDE + AEGJ + BEF] + DEFG 

J ADF + BDG + CDH + ABCE + AEGH + BEFH + CEFG 
AB FG + CEJ + DEH + ACGH + ADGJ + BCFH + BDFJ 
AC FH + BEJ + DEG + ABGH + ADHJ + BCFG + CDFJ 
AD FJ + BEH + CEG + ABGJ + ACHJ + BDFG + CDFH 
AE BCJ + BDH + CDG + GHJ + BEFG + CEFH + DEFJ 

AF BG +CH+DJ 

AG BF + CDE + EHJ + ABCH + ABDJ + CFGH + DFGJ 
AH CF + BDE + EGJ + ABCG + ACDJ + BFGH + DFHJ 

AJ DF + BCE + EGH + ABDG + ACDH + BFGJ + CFHJ 
BC GH + AEJ + DEF + ABFH + ACFG + BDHJ + CDGJ 
BD GJ + AEH + CEF + ABFJ + ADFG + BCHJ + CDGH 
BE ACJ + ADH + CDF + FHJ + AEFG + CEGH + DEGJ 
BH CG + ADE + EFJ + ABCF + AFGH + BCDJ + DGHJ 

BJ DG + ACE + EFH + ABDF + AFGJ + BCDH + CGHJ 
CD HJ + AEG + BEF + ACFJ + ADFH + BCGJ + BDGH 
CE ABJ + ADG + BDF + FGJ + AEFH + BEGH + DEHJ 

CJ DH + ABE + EFG + ACDF + AFHJ + BCDG + BGHJ 
DE ABH + ACG + BCF + FGH + AEFJ + BEGJ + CEHJ 

EF BCD + BHJ + CGJ + DGH + ABEG + ACEH + ADEJ 
EG ACD + AHJ + CEJ + DFH + ABEF + BCEH + BDEJ 

EH ABD + AGJ + BFJ + DFG + ACEF + BCEG + CDEJ 

EJ ABC + AGH + BFH + CFG + ADEF + BDEG + CDEH 
AEF BEG + CEH + DEJ + ABCD + ABHJ + ACGJ + ADGH + BCFJ + BDFH + CDFG + FGHJ 
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APPENDIX E. SAMPLE RAW RESULTS FOR 2”* PILOT RUNS 





BFR 


-0.111 
0.111 
-1.000 
-0.500 
-1.000 
0.250 
0.333 


-0.083 
-1.000 
-0.500 
0.111 
0.333 
0.111 
0.333 


-0.111 
-0.333 
-0.333 
-0.333 
-0.500 
0.111 


-0.083 
0.222 
0.000 
0.111 
0.250 
0.111 


-0.083 
-1.000 
0.250 
0.222 
0.111 


-0.111 
-0.111 
0.111 


-1.000 
-0.500 
0.111 


-0.500 
-1.000 
0.000 
0.111 


-0.500 
0.111 
0.250 
0.111 
0.000 


-0.333 
0.111 


-0.333 
-0.333 
-0.083 
0.111 


-0.083 
0.222 


-0.500 
0.111 
0.278 


-1.000 
-0.083 





AA3 





AA2 





AAL 








EXP 





C2S 





C2P 

















run 

































































20 
21 








22 
23 








24 
25 
26 
27 
28 
29 
30 
31 
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BFR 


0.111 
0.333 
0.111 
0.333 
0.111 
0.111 
0.111 
-1.000 
-0.500 
0.222 
-0.500 
0.333 
-0.083 
0.111 
-0.500 
-1.000 
0.000 
0.111 
0.000 
0.333 


-0.333 
-0.333 
0.111 


-0.167 
0.111 
0.000 
0.111 


-0.500 
-1.000 
0.333 
0.111 
0.333 


-1.000 
-0.083 
0.111 
0.000 
0.111 





AA3 





AA2 





AAL 








EXP 





C2S 





C2P 

















run 
28 
29 
30 
31 
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APPENDIX F. S-PLUS CODE FOR ANALYSIS OF 2”* DESIGN 
RESULTS 


function(res, returnDesign = F, runs=32) { 

#NAME Results 

#AUTHOR Posadas 

# 

#ARGUMENTS 

#res is a data.frame of results factors are 0O’s and 1’s 

#res includes 9 factor coloumns and one response columns (10th col) 

# 

#returnDesign is a boolean that indicates whether the factorial design 
#is to be returned by the function 

# 

#DESCRIPTION 

#the factors are converted to +,- signs, an anova is performed, then 
#the factor effects are determined along with their standard error 

# 

#RETURNS 

#the anova summary, the mean response and its standard error, the 
#factor effects & standard errors, and the design matrix, if requested 
# 

#CREATE AND LABEL FACTORS 


sig <- a(uen, way 

con <- function(x) {2 - x} 
des <- res[, 1:9] 

des[, 1] <- sig[con(des[, 1 
des[, 2] <- sig[con(des[, 2 
des[, 3] <- sig[con(des[, 3 
des[, 4] <- sig[con(des[, 4 
des[, 5] <- sig[con(des[, 5 
des[, 6] <- sig[con(des[, 6 
des[, 7] <- sig[con(des[, 7 
des[, 8] <- sig[con(des[, 8 
des[, 9] <- sig[con(des[, 9 























ll 


fnames <- list(E = sig, R sig, B = sig, C2P = sig, C2S = sig, 
EXP = sig, AAl = sig, AA2 = sig, AA3 = sig) 
factor.names(des) <- fnames 
BFR <- ressBFR # 
#CONDUCT ANOVA 
dat <- cbind(des, BFR) 
dimnames (dat) [[2]] [10] <- "BFR" 
dat.aov <- aov(BFR ~ ., data = dat) 
summ <- summary (dat.aov) # 
#MEAN AND FACTOR EFFECTS WITH STANDARD ERROR 
N <- dim(res) [1] 
BFRmean <- dat.aovscoef [1 
BFRse <- sqrt (summary (dat.aov) $Mean [10] /N) 
mean <- list(estimate = BFRmean, se = BFRse) 





effects <- model.tables(dat.aov, type = "feffects", se = T) # 
#EXTRACT DESIGN 

Design <- des[l:runs, 1:9 #RETURNS 

if (returnDesign == T) return(Design, summ, mean, effects) 


else return(summ, mean, effects) 
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APPENDIX G. S-PLUS CODE FOR POWER CALCULATIONS 


function(var, treat, pow, taul, alpha = 0.05, runs = 1) 


{ 
# 


#AUTHOR LLOYD BROWN, 2000 (MODIFIED BY POSADAS 2001) 
# 
#ARGUMENTS 


se 


var is an estimate of the variance in the data 

treat is the number of treatments 

pow is the maximum power considered 

tau is the minimum detectable departure from the mean 
alpha significance level 

runs is the number of run sets per replication 
RETURNS 


a matrix of tau (detectable departure from the mean) and 
( 


# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# 
# xX 
# m (number of replications required) 
# 
#INITIALIZE VALUES 
points <- 16 
xX <- matrix(nrow = points, ncol = 2, dimnames = list (NULL, 
c("deviation", "replications") )) 
m<- 2 
tau <- taul 
powerl <- 0 # 
# 
#CALCULATE NUMBER OF REPLICATION REQUIRED 
for(i in 1:points) { 
while (powerl < pow) { 
lambda <- (m * treat * tau*2)/var #NON-CENTRALITY 





PARAMETER 
dofl <- treat - 1 
dof2 <- treat * runs * (m - 1) 
cp <- qf(1 - alpha, dofl1, dof2) #CRITICAL POINT 
powerl <- 1 - pf(cp, dof1l, dof2, lambda) 
m<- m+ii1 
} 
x[i, 1] <- tau 
x[i, 2] <- m 
m <- 2 
tau <- tau + taul/10 
powerl <- 0 
} 
x 
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APPENDIX H. 2° FULL FACTORIAL DESIGN 
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| RunOrder | Decision Factors | Commander Attributes | 








































































































| | Environ Red Blue | C2Philos | C2Style Exper Level] 
33 - - - : + 
34 + : : : : + 
35 - + ; - E + 
36 + + ; ; : + 
37 : z + - : + 
38 + : + 2 : + 
39 - + + = a + 
40 + + + - - + 
4 : : : + : + 
42 + : 7 + 7 + 
43 - + 7 + 4 + 
44 + + 2 + é + 
45 - - + + 5 + 
46 + : + + : + 
47 - + + + - + 
48 + + + + - + 
49 - - - - + + 
50 + = = + + 
51 : + : + + 
52 + + : + + 
53 - - + ; + + 
54 + 5 + : + + 
55 - + + ; + + 
56 + + + - + + 
57 = = 2 + + + 
58 + : - + + + 
59 - + - + + + 
60 a + 7 + + + 
61 - 7 + + + + 
62 + : + + + + 
63 - + + + + + 
64 + + + + + + 
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APPENDIX I. FULL FACTORIAL ESTIMATES & POWER 
CURVES 


Pilot Run Estimates for 2° Fractional Factorial Design: 


Ul = 0.059, Oo = 0.004 = 0.000016, a= 0.01 


For a power of 0.99, using thirty replications detects a three-percent or greater deviation 


from the mean. 





80 


Power = 0.99 


60 


Replications Required 
40 


20 





0.0005 0.0010 0.0015 0.0020 
tau = Deviations from Mean Battle Force Ratio 











Power Curves for the 2° Fractional Factorial Design 
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APPENDIX J. FULL FACTORIAL RESULTS 
for CAS Periods of 2 Through 4 Hours 





CAS PERIOD = 2 HRS 















































ANOVA 

D£ Sum of Sq Mean Sq F Value Pr (F) 

E 1 0.4687 0.46875 5.6461 0.0175930 

R if 1.1779 1.17788 14.1876 0.0001705 

B a 3.5021 3.50208 42.1826 0.0000000 

C2P 1 49.0525 49.05249 590.8380 0.0000000 

c2s 1 9.8550 9.85496 118.7032 0.0000000 

EXP 1 40.1235 40.12348 483.2879 0.0000000 

E:R 1 0.0058 0.00579 0.0697 0.7917955 

E:B il 0.1565 0.15648 1.8848 0.1699493 

E:C2P 1 0.4618 0.46183 5.5628 0.0184473 

E:C28 1 0.2676 0.26759 3.2232 0.0727625 

E:EXP 1 0.7259 0.72593 8.7438 0.0031449 

R:B 1 0.7698 0.76978 9.2720 0.0023588 

R:C2P 1 0.8241 0.82410 9.9263 0.0016547 

R:C28 i 0.2521 0.25208 3.0363 0.0815809 

R:EXP 1 0.1525 0.15249 1.8368 0.1754874 

B:C2P a: 2.7000 2.70000 32.5215 0.0000000 

B:C2s 1 0.0544 0.05442 0.6555 0.4182417 

B:EXP ‘i 1.4815 1.48148 17.8445 0.0000251 

C2P:C2s 1 9.4454 9.44537 113.7696 0.0000000 

C2P:EXP 1 32.1483 32.14825 387.2262 0.0000000 

C2S:EXP a 6.8481 6.84815 82.4860 0.0000000 

Residuals 1898 157.5756 0.08302 
TWO-FACTOR INTERACTIONS 
Effects se 
E:R -0.0034722 0.013152 
MAIN EFFECTS E:B 0.0180556 0.013152 
E:C2P -0.0310185 0.013152 
E:C2S -0.0236111 0.013152 
Effects Re E:EXP -0.0388889 0.013152 
By ote Oe ee Oe eae R:B -0.0400463 0.013152 
Bees F002 OFS LSe R:C2P 0.0414352 0.013152 
BA Muy ORSe Les. elle R:C2S -0.0229167 0.013152 
C2E = Ura 8 12240 00S 152 R:EXP 0.0178241 0.013152 
Rae Sie ee COs Oe De B:C2P -0.0750000 0.013152 
BRP (On 282 L208 Oe OES lee B:C2S 0.0106481 0.013152 
B:EXP -0.0555556 0.013152 
C2P:C28 0.1402778 0.013152 
C2P:EXP -0.2587963 0.013152 
C2S:EXP 0.1194444 0.013152 
BFR 
Mean = 0.069 


Standard Error = 


0.007 
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CAS PERIOD = 3 HRS 


























ANOVA 

D£ Sum of Sq Mean Sq F Value Pr (F) 

E 1 0.7787 0.77870 9.8270 0.0017460 

R 1 1.7120 1.71204 21.6053 0.0000036 

B 1 5.0704 5.07037 63.9862 0.0000000 

C2P 1 38.2819 38.28189 483.1033 0.0000000 

C28 if 5.7300 5.73004 72.3110 0.0000000 

EXP 1 35.9951 35.99509 454.2447 0.0000000 

E:R 1 0.1087 0.10867 1.3713 0.2417278 

E:B 1 0.0391 0.03912 0.4937 0.4823745 

E:C2P 1 0.3766 0.37657 4.7522 0.0293840 

E:C2S 1 0.0093 0.00928 0.1172 0.7321590 

E:EXP ‘ 0.6100 0.60998 7.6977 0.0055832 

R:B 1 0.1155 0.11546 1.4570 0.2275529 

R:C2P ” 1.2790 1.27904 16.1410 0.0000611 

R:C2S it 0.3169 0.31690 3.9991 0.0456659 

R:EXP 1 0.6421 0.64208 8.1028 0.0044672 

B:C2P 1 4.3447 4.34468 54.8282 0.0000000 

B:C2S i 0.0058 0.00579 0.0730 0.7870041 

B:EXP 1 2.9384 2.93837 37.0812 0.0000000 

C2P:C28 1 6.3531 6.35311 80.1739 0.0000000 

C2P:EXP 1 26.3412 26.34115 332.4156 0.0000000 

C2S:EXP it 4.1979 4.19794 52.9765 0.0000000 

Residuals 1898 150.4006 0.07924 
TWO-FACTOR INTERACTIONS 
Effects se 
E:R -0.0150463 0.012849 
MAIN EFFECTS E:B -0.0090278 0.012849 
E:C2P -0.0280093 0.012849 
E:C2S -0.0043981 0.012849 
Effects se E:EXP -0.0356481 0.012849 
Bo Wie QO2 (cen leet ses R:B -0.0155093 0.012849 
Roel) 2) 2e2 05002889 R:C2P 0.0516204 0.012849 
BO. et te O.OiZee2 R:C28 -0.0256944 0.012849 
GAP Weegee Os Oe0l 2849 R:EXP 0.0365741 0.012849 
Ree Se Ua eee ae B:C2P -0.0951389 0.012849 
BAD “Oe 8428 0. OlZ842 B:C2S 0.0034722 0.012849 
B:EXP -0.0782407 0.012849 
C2P:C28 0.1150463 0.012849 
C2P:EXP -0.2342593 0.012849 
C2S:EXP 0.0935185 0.012849 
BFR 


Mean = 0.106 


Standard Error = 


0.006 
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CAS PERIOD = 4 HRS 


























ANOVA 

D£ Sum of Sq Mean Sq F Value Pr (F) 

E il 0.5150 0.51498 7.1546 0.0075414 

R 1 1.0340 1.03396 14.3649 0.0001553 

B 1 3.6265 3.62655 50.3840 0.0000000 

Cc2P 1 32.6969 32.69692 454.2614 0.0000000 

c2s 1 6.1880 6.18802 85.9708 0.0000000 

EXP 1 30.6985 30.69846 426.4967 0.0000000 

E:R 1 0.3612 0.36117 5.0178 0.0252040 

E:B 1 0.0911 0.09106 1.2650 0.2608401 

E:C2P 0.3735 0.37346 5.1886 0.0228468 

E:C28 1 0.0008 0.00078 0.0108 0.9172058 

E:EXP if 0.5150 0.51498 7.1546 0.0075414 

R:B 1 0.2809 0.28087 3.9022 0.0483687 

R:C2P 1 1.0032 1.00325 13.9382 0.0001945 

R:C28 a 0.3140 0.31405 4.3631 0.0368579 

R:EXP a 0.4515 0.45155 6.2734 0.0123395 

B:C2P 1 3.0171 3.01714 41.9174 0.0000000 

B:C28 1 0.1315 0.13149 1.8268 0.1766701 

B:EXP 1 2.1259 2.12593 29.5358 0.0000001 

C2P:C28 1 6.4687 6.46868 89.8700 0.0000000 

C2P:EXP 1 22.0306 22.03061 306.0734 0.0000000 

C2S:EXP 1 4.7225 4.72254 65.6107 0.0000000 

Residuals 1898 136.6146 0.07198 
TWO-FACTOR INTERACTIONS 
Effects se 
E:R -0.0274306 0.012246 
MAIN EFFECTS E:B 0.0137731 0.012246 
E:C2P -0.0278935 0.012246 
E:C28 0.0012731 0.012246 
Effects a E:EXP -0.0327546 0.012246 
Br 0027826) 0 OT2245 R:B -0.0241898 0.012246 
Riese e era: Pvt eess R:C2P 0.0457176 0.012246 
Bo Deer 2is 0 o0t eee R:C2S -0.0255787 0.012246 
O2F Peer ea eee R:EXP 0.0306713 0.012246 
G25 teas tO 0T ees B:C2P -0.0792824 0.012246 
EAEY  Veene Suse Oe Ol 22k6 B:C28 0.0165509 0.012246 
B:EXP -0.0665509 0.012246 
C2P:C28 0.1160880 0.012246 
C2P:EXP -0.2142361 0.012246 
C2S:EXP 0.0991898 0.012246 
BFR 


Mean = 0.142 


Standard Error = 


0.006 
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CAS PERIOD =5 HRS 





ANOVA 


Df Sum of Sq Mean Sq F Value 








E 1 0.3461 0.34609 5.0823 

R Ak 0.6340 0.63398 9.3098 

B 1 5.9259 5.92593 87.0207 

C2P 1 28.3025 28.30249 415.6150 

C2s 1 5.2547 5.25473 77.1644 

EXP 1 27.7120 27.71204 406.9443 

E:R 1 0.5942 0.59424 8.7262 

E:B 1 0.0568 0.05682 0.8343 
E:C2P 1 0.2676 0.26759 3.9295 
E:C2S 1 0.1225 0.12245 1.7982 
E:EXP a 0.2729 0.27287 4.0070 
R:B 1 0.1053 0.10535 1.5470 
R:C2P 1 0.3642 0.36422 5.3485 
R:C2S 1 0.0004 0.00041 0.0060 
R: EXP 1 0.3704 0.37037 5.4388 
B:C2P 1 4.4083 4.40833 64.7353 
B:C2S 1 0.0021 0.00208 0.0306 
B:EXP 1 4.4297 4.42966 65.0484 
C2P:C2S 1. 4.9794 4.97942 73.1216 
C2P:EXP 1 21.8643 21.86430 321.0718 
C2S:EXP 1 3.3891 3.38912 49.7684 





Residuals 1898 129.2497 0.06810 


COOCOCOCOOCOCOOCOOOOOOCOOOAOa Oo 


Pr (F) 


-0242855 
- 0023109 
- 0000000 
- 0000000 
- 0000000 
- 0000000 
- 0031752 
- 3611405 
- 0475892 
-1800905 
- 0454549 
-2137272 
- 0208463 
-9380451 
- 0197984 
- 0000000 
- 8611696 
- 0000000 
- 0000000 
- 0000000 
- 0000000 











E:R 

MAIN EFFECTS Eee 

E:C2P 

E:C2S 

Effects se F:EXP 

E 0.02685185 0.011911 R:B 
Shee re Rak 

i Le 2 ReC25 

C2P 0.24282407 0.011911 R:EXP 
C2S -0.10462963 0.011911 B:C2P 
EXP 0.24027778 0.011911 B:C28 
B: EXP 

C2P:C2S 

C2P:EXP 

C2S:EXP 





Effects 


.03518519 
.01087963 
-02361111 
-01597222 
-02384259 
-01481481 
- 02754630 
-00092593 
-02777778 
- 09583333 
. 00208333 
-09606481 
-10185185 
- 21342593 
- 08402778 


COOCOOCOOOCOOOCOOAOAAO0O fo 
CoOOCOOCOOOCOOOOAOAOAAOA0O fo 


TWO-FACTOR INTERACTIONS 


se 
ETO DA: 
11911 
11911 
11971: 
11911 
11927 
11911 
11911 
11911 
11911 
11911 
L911. 
11911 
11911: 
11911 














BFR 


Mean = 0.162 


Standard Error = 0.006 
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CAS PERIOD = 6 HRS 



































ANOVA 

D£ Sum of Sq Mean Sq F Value Pr (F) 

E il 0.2890 0.28899 4.5840 0.0323990 

R 1 1.3021 1.30208 20.6537 0.0000058 

B 1 4.2188 4.21875 66.9179 0.0000000 

Cc2P 1 19.3782 19.37819 307.3772 0.0000000 

c2s 1 3.8920 3.89200 61.7350 0.0000000 

EXP 1 22.0544 22.05442 349.8277 0.0000000 

E:R 1 0.0021 0.00208 0.0330 0.8557707 

E:B a) 0.0021 0.00208 0.0330 0.8557707 

E:C2P 0.1408 0.14084 2.2341 0.1351646 

E:C28 1 0.2321 0.23212 3.6820 0.0551539 

E:EXP if 0.1486 0.14856 2.3565 0.1249320 

R:B 1 0.0202 0.02016 0.3199 0.5717638 

R:C2P 1 0.8803 0.88027 13.9629 0.0001920 

R:C28 a 0.0026 0.00257 0.0408 0.8399512 

R:EXP a 0.7346 0.73459 11.6521 0.0006548 

B:C2P 1 2.2994 2.29941 36.4733 0.0000000 

B:C28 1 0.1992 0.19918 3.1593 0.0756534 

B:EXP 1 3.1687 3.16875 50.2628 0.0000000 

C2P:C28 1 4.3447 4.34468 68.9153 0.0000000 

C2P:EXP 1 14.4676 14.46759 229.4852 0.0000000 

C2S:EXP 1 2.9558 2.95579 46.8848 0.0000000 

Residuals 1898 119.6569 0.06304 
TWO-FACTOR INTERACTIONS 
Effects se 
E:R -0.0020833 0.01146 
MAIN EFFECTS E:B 0.0020833 0.01146 
E:C2P -0.0171296 0.01146 
E:C28 0.0219907 0.01146 
Effects ee E:EXP -0.0175926 0.01146 
Be CV 0as ws ee R:B -0.0064815 0.01146 
Rests 0b2 0002) 0. 0TTSS R:C2P 0.0428241 0.01146 
Bee ee eee R:C28 -0.0023148 0.01146 
CAP 0, 2009257, DOr e6 R:EXP 0.0391204 0.01146 
C28. “OO 200463). 0s -0TT6 B:C2P -0.0692130 0.01146 
BEP Deets set ORO LreS B:C2S 0.0203704 0.01146 
B:EXP -0.0812500 0.01146 
C2P:C28 0.0951389 0.01146 
C2P:EXP -0.1736111 0.01146 
C2S:EXP 0.0784722 0.01146 
BFR 


Mean = 0.195 


Standard Error = 


0.006 
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APPENDIX K. SSIM CODE JAVA CLASSES 


SSIM CODE consists of a base java package with test classes and an evaluation 
package with test classes. The base package is comprised of over 2,600 lines of Java. Its 
test classes include over 860 lines of source. The evaluation package contains over 3,100 
lines of code, and the corresponding test classes consist of over 2,400 lines of Java. In 
total, SSIM CODE encompasses approximately 9,120 lines of Java (about half are 
documentation). The Combat XXI simulation contained over 200,000 lines of code as of 


February 2001 (version 1.02). 


A Combat XXI platform entity contains an SA module, and may contain a SSIM 
CODE commander module. The SSIM CODE commander module contains a set of 
OODA loops and a set of reports. Each OODA loop contains a decision. The following 
figures depict the key classes in SSIM CODE. The methods and attributes for each class 


are shown. 

















Platform Entity 
(Extends cod Entity, 
Deplements cood Platfonn Interface) 


Situational Awareness Module 
(hmplemerts cod Fumctionality Module) 
Cormunander 
Chuplemerts cxod Pumctionality Module) 
Report OOD S, 

Benort OOD S 
Report 
(Extends java Object) 






OoODA 
(Extends coc Event Handler Base) 








Decision 
(Extends java Object) 





Class Relationships 
The primary classes in SSIM CODE are: Commander, OODA, Report and Decision. 
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Pl CdtActions extends Object implements Actionslfc 
Pl CdiFacts extends SuperFact implements Factslfc 
PL Commander extends cxxiE ventHandlerB ase implements FunctionalityModule 
TT CommandPlatformlfe extends Platformlfc 

Pl DecideEngageActions extends Object implements Actionslfc 
Pl DecideMovedctions extends Object implements Actionslfc 
Pl DecideSearchActions extends Object implements Actionslfc 
Pl Decision extends Object 

PL EventGenerator extends cxxiE ventHandlerBase 
PI Hashtable? extends HashMap 

PT Goda extends cxxiE ventHandlerB ase 

Pl Report extends Object 

PT TestCommander extends cxxiE ventHandlerB ase 

PL TestEventGenerator extends cxxiE ventHandlerB ase 
PI TestMatrix extends Object 

PL TestOoda extends cxxiE ventHandlerB ase 

PL TestSsimCode extends Object 





SSIM CODE Base Package and Test Classes 








Et we ssimcode. eval 

nm Attack extends SimEntityB ase 

BluePlt extends Object 

CeirActions extends Object implements Actionslfc 
Cdrlntent extends Object 

Defend extends SimEntityBase implements Defendlfc 
Defendlfc 

EnemyCo extends Object 

Engageactions extends Object implements Actionslfc 
MoveActions extends Object implements Actionslfc 
NAI extends Object 

SearchActions extends Object implements Actionslfc 
TAI extends Object 

TestDefend extends SimEntityB ase 

TestScenario extends Object 

TestScenario2 extends Object 

TestScenario3 extends Object 


CRO BUEC RGEC ROB OBCECECRCROBORCgE 


ee ee eee ee EE 








SSIM CODE Evaluation Package and Test Classes 
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TL Commander extends cxxiE ventHandlerBase implements FunctionalityModule 
®@ actDelay double 
@ ¢2Philos,boolean 
@ c2Style.boolean 
®@  cdilntentE ror, double 
®@ cdType,.CommanderT ype 
@) debug.boolean 
@  decideDelay double 
®@ deciding.boolean 
@ decisionDistribution Hashtable[] 
®@ decisionQueue,Hashtable 
@ decisionsVector 
@ decisionTypes.¥Vector 
@  oexperienceLevel int 
®@ name.String 
@ observeDelay,double 
®@ ocodaDelay double 
W@ oodaLoops,Hashtable 
@  ofientDelay,double 
@ platformEntitylfc 
@ randStream,cxxiRandom 
@ reports Hashtable 
@ saMod, Sit4wareModule 
@  startDelay double 























SSIM CODE Commander Attributes 


TL Goda extends cxxiE ventHandlerBase 
active,boolean 
blueDF,String 

blueV alue,double 
cdr,Commander 
combinedDf,String 
debug.boolean 
decision,Decision 
decisionT ype,String 
environDF,String 
environ mF actor,double 
environ¥alue,double 
name,String 
outcome,String 
outcomeProb,double 
redDF,.String 
red¥alue,double 








SSIM CODE OODA Attributes 
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TT Commander extends cxxiE ventHandlerBase implements FunctionalityModule 


(@) Commander(String, CommanderT ype. int, boolean, boolean, FunctionalityModuleHolder) 
@) Commander{ } 

@) Commander(Commander) 

@) doEndDecisionCycle[Decision), void 

WW) doRequestDecision(Object),void 

@) doStartDecisionCycle(Decision).vaid 

W@) getActDelayl ).double 

®) getC2Philos{ ,boolean 

@ getC2Stylef j.boolean 

@) getCdrlntentE ror ).double 

@) getCommanderT ypef ),CommanderT ype 
(@) getDebug{ J.boolean 

(@) getDecideDelayl },double 

) getDecisionDistribution(S tring) Hashtable 
W@W) getDecisionEnumeration{String),Enumeration 
W@) getDecisionProbability(S tring, String),double 
(@) getDecisionQueue{ J, Hashtable 

(@) getDecisions{ )Vector 

@ getDecisionTypes{ |.vector 

@) getDetails{ ). String 

() getExperienceLevel{ J.int 

getName, ].String 

(@) getObserveDelayl }, double 

(@) getOodaDelayl ), double 

(@) getOodaLoops{ ), Hashtable 

WW) getOrientDelay{ ).double 

WW getPlatform{ ).Entitylfc 

(@) getRandStream| ).cxxiRandom 

(@) getReports{ | Hashtable 

@) getSeed J.long 

W) getSitAwareModule{ ].SitAwareModule 

W@W) getTypef J.int 

@ isAgaressive[ |. boolean 

®) isConservative{ },boolean 

®@) isDecidingf ).boolean 

@) isDetailed{ ),boolean 

@ isMission{ ).boolean 

@ registerDecisionCycles{ J. void 

@ requestGuidancef ).void 

® reset( ).void 

® tesolveDecision{ J, void 

@ tettiveMission{ |.void 

(®) reviewMission{ ),void 

@ scheduleEndDecisionCycle(Decision).void 
@) scheduleRequestDecision(Object), void 
(@ scheduleRequestD ecisionD elay(Object[]]. void 
WW scheduleStartDecisionCycle(Decision],void 
W@) setActDelay(double).void 

@  setAliDecisionDistributions{ J.void 

® setC2Philos{boolean), void 

@ setC2Style(boolean), void 

® setCdlntentError(double),void 

(®) setCommanderl ype(CommanderT ype). void 
(@) setDebug(boolean), void 

@) setDecideDelay(double),void 

@) setDeciding[boolean), void 

@ setDecisionDistribution{¥ector, boolean),void 
@) setDecisionProbability(String, String, double), void 
@  setDecisionQueuef ).void 

®) setExperienceLevelfint), void 

® setName(String),void 

@® setObserveDelay(double), void 

(@) setOodaDelay(double), void 

@) setOrientDelay(double).void 

@® setPlatform{(FunctionalityM oduleHolder).void 
@®) setRandStream(cxxiR andom),void 

@ setReports{ J.void 

@  setSitAwareModule(SitAwareModule), void 
@ setStartDelay(double),void 

@ startDeciding[String), void 








WW) toString{ ).String 
SSIM CODE Commander Methods 
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PL Ooda extends cxxiE ventHandlerBase 











M) doAct{ |void 

) doDecidef ).void 

) doEndDecisionCycle[D ecision), void 
() doEngageAction{ ).void 

@) dolmproveReports{ J. void 

() doMovesction{ ).void 

) doObservel ).void 

) doDrient( J, void 

() doSearchAction{ J, void 

®) doStartDecisionCycle(Decision),void 
(®) getBayesOutcomeProbf },double 
®) getCombinedDf{ ).String 

@®) getCommander{ }, Commander 

®) getDebua ).boolean 

®) getDecision{ |Decision 

@) getDecisionTypef },String 

() getMCOutcomeProb{ ),double 
W@) getOutcomes ).String 

®) isActive[ ).boolean 

@ Ooda(Commander, String) 

@ reset[ J.void 

®) scheduleAct( ).void 

W@) scheduleDecide{ ).void 

®) scheduleEndDecisionCycle{ ).void 
@ scheduleEngageAction{ ).void 

@) schedulelmproveReports{ ).void 
@ scheduleMovedctionf J, void 

@) scheduleObservef |.void 

®) scheduleOrient{ J. void 

W@W) scheduleSearchAction{ J, void 

@) scheduleStartDecisionCycle(Decision),void 
@  setActive(boolean),void 

@  setCommander(Commander),void 
@ setDebug{boolean). void 

@) setDecision(Decision), void 

@ setDecisionType(String), void 

@ setOutcome(String),void 

(@) toStringf ),String 
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TL Report extends Object 
(  cdi.Commander 
(@) debug_boolean 
@ name.String 
@ plat,Platformlfc 
@ reportStatus,Object[] 
WW) getD ebugi ),boolean 
) improveReport(double), Object] 
(@) Report(Commander, String) 
(®) setDebug{boolean). void 
() setEnemyForceStateReport(int, double),void 
(@) setEnvironmentStateReport(int, double),void 
(@) setOwnForceStateReport(int, double), void 
(@) setReport(int, double), void 
(W) toStrinal },String 
(@) updateReport( ),Object{] 
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TT Decision extends Object 
edr,Commander 
complete,boolean 
debug.boolean 
decisionE ndT ime,double 
decisionF actors,Object[] 
decisionRequestT ime,double 
decisionStartT ime,double 
decisionT ime,double 

decisionT ype,String 

ooda,Qoda 

outcome,String 
outcomeProbability,double 
randDraw,double 

Decision{ ] 

®) Decision{String, Commander) 

@) getCdr ).Commander 

@) getDebugi ),boolean 

@) getDecisionEndT imef },double 
®) getDecisionFactors{ ),Object[] 
@) getDecisionRequestT ime ),double 
@) getDecisionStartTimel ).double 
®) getDecisionT imef ,double 

®) getDecisionTypef ).String 

®) getDetails{ },String 

®) getOoda }.Ooda 

@) getOutcomef ).String 

®) getOutcomeProbability[ }. double 
®) getRandomDraw{ ).double 

®) isCompletef },boolean 

®@  setCdil[(Commander),void 

@  setComplete(boolean),void 

@ setDebug({boolean). void 

@) setDecisionEndT ime{double),void 
@) setDecisionFactors(Object[]).void 
@) setDecisionRequestT ime[double),void 
@) setDecisionStartT ime{double),void 
®@ setDecisionType(String),void 

®) setOoda(Ooda),void 

@ setOutcome(String), void 

@) setOutcomeProbability(double),void 
@) setRandomDraw(double),void 

@) toStrinaf String 
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B  TestScenario.java 














Pl TestScenario extends Object 


ccit4ction,CcirActions 

debug.boolean 

engageAction,EngageActions 

fa.int{][] 

HOLD_TM_.double.final 

moveAction MovedActions 

randStream,cxxiR andom 

REPLICATIONS. int final 

RUNS. int.final 

search4ction,SearchActions 

addActionsF acts(Commander),void 
checkFactsActions[boolean, boolean, Platformlfc).void 
configureBlueCommande:r(Commander. int[]). void 
configureBlueD efense[D efend, Commander). ¥oid 
configurePlatform{ ).Platformlfc 
configureRedAttack[Attack, int[]). void 
createBlueCommander(Platformlfc, int[]}, Commander 
createBlueDefense[Attack, double),.Defend 
createFactorArrays{ ).int([][] 

createRedattack[ | Attack 

dumpResults(int[], Defend).void 

getD ebug[ ),boolean 

main{String[]). void 

prepOutputFile{ ),Printriter 

prepSimf J. void 

returnResults(int[], Defend),String 

setD ebug[boolean). void 

setFactors{ ).int[] 

setFactors{int[]).void 

setReports(int[], Commander). void 

singleRuntint[]).S tring 


startSim[boolean, Commander, Platformlfc, Attack, Defend), void 


summarize(Commander),void 
TestS cenario(cxxiR andom) 
updateSA([Object. Platformlfc). void 
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